2026-02-18T08:47:22.483 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-18T08:47:22.497 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-18T08:47:22.573 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/skanta-2026-02-18_08:32:17-rados-wip-bharath18-testing-2026-02-13-0856-distro-default-trial/55841 branch: wip-bharath18-testing-2026-02-13-0856 description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} email: skanta@redhat.com first_in_suite: false flavor: default job_id: '55841' kernel: branch: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: trial name: skanta-2026-02-18_08:32:17-rados-wip-bharath18-testing-2026-02-13-0856-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-bharath18-testing-2026-02-13-0856 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 325850157d309f65985eea9ec9bb9f1d56726fee ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 325850157d309f65985eea9ec9bb9f1d56726fee workunit: branch: wip-bharath18-testing-2026-02-13-0856 sha1: 325850157d309f65985eea9ec9bb9f1d56726fee owner: scheduled_skanta@soko04.front.sepia.ceph.com priority: 60 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 7109 sha1: 325850157d309f65985eea9ec9bb9f1d56726fee sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath18-testing-2026-02-13-0856 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 325850157d309f65985eea9ec9bb9f1d56726fee targets: trial118.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPuv5Ta0OeqiA4P/2GoABcgxjNzDdZGTJ99vPcF78hXhPA7UGRk7W8ovlJxWJvnWSgrBbSbl1Hf+i5BGF9+Q4L4= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - misc teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-18_08:32:17 tube: trial user: skanta verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682 2026-02-18T08:47:22.573 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa; will attempt to use it 2026-02-18T08:47:22.574 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa/tasks 2026-02-18T08:47:22.574 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-18T08:47:22.575 INFO:teuthology.task.internal:Checking packages... 2026-02-18T08:47:22.575 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '325850157d309f65985eea9ec9bb9f1d56726fee' 2026-02-18T08:47:22.575 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-18T08:47:22.575 INFO:teuthology.packaging:ref: None 2026-02-18T08:47:22.575 INFO:teuthology.packaging:tag: None 2026-02-18T08:47:22.575 INFO:teuthology.packaging:branch: wip-bharath18-testing-2026-02-13-0856 2026-02-18T08:47:22.575 INFO:teuthology.packaging:sha1: 325850157d309f65985eea9ec9bb9f1d56726fee 2026-02-18T08:47:22.575 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-bharath18-testing-2026-02-13-0856 2026-02-18T08:47:22.701 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5271-g32585015-1jammy 2026-02-18T08:47:22.702 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-18T08:47:22.713 INFO:teuthology.task.internal:no buildpackages task found 2026-02-18T08:47:22.713 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-18T08:47:22.727 INFO:teuthology.task.internal:Saving configuration 2026-02-18T08:47:22.735 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-18T08:47:22.746 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-18T08:47:22.824 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial118.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/skanta-2026-02-18_08:32:17-rados-wip-bharath18-testing-2026-02-13-0856-distro-default-trial/55841', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-18 08:43:52.440137', 'locked_by': 'scheduled_skanta@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMp/+A2mKunBot5RnIh4YUpQwNAY+B6SnZuKFvkezPkx'} 2026-02-18T08:47:22.825 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-18T08:47:22.836 INFO:teuthology.task.internal:roles: ubuntu@trial118.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-18T08:47:22.836 INFO:teuthology.run_tasks:Running task console_log... 2026-02-18T08:47:22.922 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x775b7d1d8ae0>, signals=[15]) 2026-02-18T08:47:22.922 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-18T08:47:22.933 INFO:teuthology.task.internal:Opening connections... 2026-02-18T08:47:22.933 DEBUG:teuthology.task.internal:connecting to ubuntu@trial118.front.sepia.ceph.com 2026-02-18T08:47:22.933 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-18T08:47:22.995 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-18T08:47:23.006 DEBUG:teuthology.orchestra.run.trial118:> uname -m 2026-02-18T08:47:23.073 INFO:teuthology.orchestra.run.trial118.stdout:x86_64 2026-02-18T08:47:23.073 DEBUG:teuthology.orchestra.run.trial118:> cat /etc/os-release 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:NAME="Ubuntu" 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:VERSION_ID="22.04" 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:VERSION_CODENAME=jammy 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:ID=ubuntu 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:ID_LIKE=debian 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-18T08:47:23.117 INFO:teuthology.orchestra.run.trial118.stdout:UBUNTU_CODENAME=jammy 2026-02-18T08:47:23.118 INFO:teuthology.lock.ops:Updating trial118.front.sepia.ceph.com on lock server 2026-02-18T08:47:23.205 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-18T08:47:23.217 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-18T08:47:23.228 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-18T08:47:23.228 DEBUG:teuthology.orchestra.run.trial118:> test '!' -e /home/ubuntu/cephtest 2026-02-18T08:47:23.232 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-18T08:47:23.243 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-18T08:47:23.243 DEBUG:teuthology.orchestra.run.trial118:> test -z $(ls -A /var/lib/ceph) 2026-02-18T08:47:23.278 INFO:teuthology.orchestra.run.trial118.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-18T08:47:23.279 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-18T08:47:23.365 INFO:teuthology.run_tasks:Running task kernel... 2026-02-18T08:47:23.385 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-18T08:47:23.385 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-18T08:47:23.385 DEBUG:teuthology.orchestra.run.trial118:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-18T08:47:23.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-18T08:47:23.388 DEBUG:teuthology.orchestra.run.trial118:> uname -r 2026-02-18T08:47:23.432 INFO:teuthology.orchestra.run.trial118.stdout:5.15.0-164-generic 2026-02-18T08:47:23.432 INFO:teuthology.task.kernel:Running kernel on trial118: 5.15.0-164-generic 2026-02-18T08:47:23.432 DEBUG:teuthology.orchestra.run.trial118:> sudo apt-get clean 2026-02-18T08:47:23.494 DEBUG:teuthology.orchestra.run.trial118:> sudo apt-get update 2026-02-18T08:47:23.951 INFO:teuthology.orchestra.run.trial118.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-18T08:47:24.024 INFO:teuthology.orchestra.run.trial118.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-18T08:47:24.302 INFO:teuthology.orchestra.run.trial118.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-18T08:47:24.425 INFO:teuthology.orchestra.run.trial118.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-18T08:47:24.540 INFO:teuthology.orchestra.run.trial118.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3240 kB] 2026-02-18T08:47:24.871 INFO:teuthology.orchestra.run.trial118.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [494 kB] 2026-02-18T08:47:24.879 INFO:teuthology.orchestra.run.trial118.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.2 kB] 2026-02-18T08:47:24.879 INFO:teuthology.orchestra.run.trial118.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-18T08:47:24.908 INFO:teuthology.orchestra.run.trial118.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-18T08:47:24.914 INFO:teuthology.orchestra.run.trial118.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5295 kB] 2026-02-18T08:47:25.042 INFO:teuthology.orchestra.run.trial118.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1001 kB] 2026-02-18T08:47:25.061 INFO:teuthology.orchestra.run.trial118.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2975 kB] 2026-02-18T08:47:25.133 INFO:teuthology.orchestra.run.trial118.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-18T08:47:25.133 INFO:teuthology.orchestra.run.trial118.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1017 kB] 2026-02-18T08:47:28.275 INFO:teuthology.orchestra.run.trial118.stdout:Fetched 16.0 MB in 2s (7218 kB/s) 2026-02-18T08:47:28.797 INFO:teuthology.orchestra.run.trial118.stdout:Reading package lists... 2026-02-18T08:47:28.811 DEBUG:teuthology.orchestra.run.trial118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-18T08:47:28.886 INFO:teuthology.orchestra.run.trial118.stdout:Reading package lists... 2026-02-18T08:47:28.995 INFO:teuthology.orchestra.run.trial118.stdout:Building dependency tree... 2026-02-18T08:47:28.996 INFO:teuthology.orchestra.run.trial118.stdout:Reading state information... 2026-02-18T08:47:29.081 INFO:teuthology.orchestra.run.trial118.stdout:The following packages were automatically installed and are no longer required: 2026-02-18T08:47:29.081 INFO:teuthology.orchestra.run.trial118.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-18T08:47:29.082 INFO:teuthology.orchestra.run.trial118.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-18T08:47:29.082 INFO:teuthology.orchestra.run.trial118.stdout: sg3-utils-udev 2026-02-18T08:47:29.082 INFO:teuthology.orchestra.run.trial118.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-18T08:47:29.082 INFO:teuthology.orchestra.run.trial118.stdout:The following additional packages will be installed: 2026-02-18T08:47:29.082 INFO:teuthology.orchestra.run.trial118.stdout: linux-generic linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-18T08:47:29.082 INFO:teuthology.orchestra.run.trial118.stdout: linux-headers-generic linux-image-5.15.0-170-generic 2026-02-18T08:47:29.082 INFO:teuthology.orchestra.run.trial118.stdout: linux-modules-5.15.0-170-generic linux-modules-extra-5.15.0-170-generic 2026-02-18T08:47:29.082 INFO:teuthology.orchestra.run.trial118.stdout:Suggested packages: 2026-02-18T08:47:29.083 INFO:teuthology.orchestra.run.trial118.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2026-02-18T08:47:29.096 INFO:teuthology.orchestra.run.trial118.stdout:The following NEW packages will be installed: 2026-02-18T08:47:29.096 INFO:teuthology.orchestra.run.trial118.stdout: linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-18T08:47:29.096 INFO:teuthology.orchestra.run.trial118.stdout: linux-image-5.15.0-170-generic linux-modules-5.15.0-170-generic 2026-02-18T08:47:29.096 INFO:teuthology.orchestra.run.trial118.stdout: linux-modules-extra-5.15.0-170-generic 2026-02-18T08:47:29.096 INFO:teuthology.orchestra.run.trial118.stdout:The following packages will be upgraded: 2026-02-18T08:47:29.097 INFO:teuthology.orchestra.run.trial118.stdout: linux-generic linux-headers-generic linux-image-generic 2026-02-18T08:47:29.459 INFO:teuthology.orchestra.run.trial118.stdout:3 upgraded, 5 newly installed, 0 to remove and 53 not upgraded. 2026-02-18T08:47:29.459 INFO:teuthology.orchestra.run.trial118.stdout:Need to get 113 MB of archives. 2026-02-18T08:47:29.459 INFO:teuthology.orchestra.run.trial118.stdout:After this operation, 585 MB of additional disk space will be used. 2026-02-18T08:47:29.459 INFO:teuthology.orchestra.run.trial118.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-170-generic amd64 5.15.0-170.180 [22.7 MB] 2026-02-18T08:47:30.557 INFO:teuthology.orchestra.run.trial118.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-170-generic amd64 5.15.0-170.180 [11.6 MB] 2026-02-18T08:47:30.828 INFO:teuthology.orchestra.run.trial118.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-170-generic amd64 5.15.0-170.180 [64.0 MB] 2026-02-18T08:47:32.362 INFO:teuthology.orchestra.run.trial118.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.170.159 [1654 B] 2026-02-18T08:47:32.362 INFO:teuthology.orchestra.run.trial118.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.170.159 [2474 B] 2026-02-18T08:47:32.362 INFO:teuthology.orchestra.run.trial118.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170 all 5.15.0-170.180 [12.4 MB] 2026-02-18T08:47:32.640 INFO:teuthology.orchestra.run.trial118.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170-generic amd64 5.15.0-170.180 [2828 kB] 2026-02-18T08:47:32.709 INFO:teuthology.orchestra.run.trial118.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.170.159 [2316 B] 2026-02-18T08:47:32.907 INFO:teuthology.orchestra.run.trial118.stdout:Fetched 113 MB in 4s (31.4 MB/s) 2026-02-18T08:47:32.928 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package linux-modules-5.15.0-170-generic. 2026-02-18T08:47:33.094 INFO:teuthology.orchestra.run.trial118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 131954 files and directories currently installed.) 2026-02-18T08:47:33.096 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../0-linux-modules-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-18T08:47:33.097 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-18T08:47:33.428 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package linux-image-5.15.0-170-generic. 2026-02-18T08:47:33.433 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../1-linux-image-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-18T08:47:33.451 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-18T08:47:33.492 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-170-generic. 2026-02-18T08:47:33.498 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-18T08:47:33.499 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-18T08:47:34.604 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../3-linux-generic_5.15.0.170.159_amd64.deb ... 2026-02-18T08:47:34.604 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking linux-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-18T08:47:34.627 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.170.159_amd64.deb ... 2026-02-18T08:47:34.627 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking linux-image-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-18T08:47:34.643 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package linux-headers-5.15.0-170. 2026-02-18T08:47:34.650 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../5-linux-headers-5.15.0-170_5.15.0-170.180_all.deb ... 2026-02-18T08:47:34.651 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-18T08:47:36.016 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package linux-headers-5.15.0-170-generic. 2026-02-18T08:47:36.028 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../6-linux-headers-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-18T08:47:36.028 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-18T08:47:36.625 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.170.159_amd64.deb ... 2026-02-18T08:47:36.625 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking linux-headers-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-18T08:47:36.651 INFO:teuthology.orchestra.run.trial118.stdout:Setting up linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-18T08:47:36.652 INFO:teuthology.orchestra.run.trial118.stdout:Setting up linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-18T08:47:36.653 INFO:teuthology.orchestra.run.trial118.stdout:Setting up linux-headers-generic (5.15.0.170.159) ... 2026-02-18T08:47:36.654 INFO:teuthology.orchestra.run.trial118.stdout:Setting up linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-18T08:47:37.446 INFO:teuthology.orchestra.run.trial118.stdout:Setting up linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-18T08:47:38.284 INFO:teuthology.orchestra.run.trial118.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-170-generic 2026-02-18T08:47:38.285 INFO:teuthology.orchestra.run.trial118.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-170-generic 2026-02-18T08:47:38.290 INFO:teuthology.orchestra.run.trial118.stdout:Setting up linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-18T08:47:39.104 INFO:teuthology.orchestra.run.trial118.stdout:Setting up linux-image-generic (5.15.0.170.159) ... 2026-02-18T08:47:39.105 INFO:teuthology.orchestra.run.trial118.stdout:Setting up linux-generic (5.15.0.170.159) ... 2026-02-18T08:47:39.107 INFO:teuthology.orchestra.run.trial118.stdout:Processing triggers for linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-18T08:47:39.112 INFO:teuthology.orchestra.run.trial118.stdout:/etc/kernel/postinst.d/initramfs-tools: 2026-02-18T08:47:39.112 INFO:teuthology.orchestra.run.trial118.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-170-generic 2026-02-18T08:47:46.184 INFO:teuthology.orchestra.run.trial118.stdout:W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast 2026-02-18T08:47:52.814 INFO:teuthology.orchestra.run.trial118.stdout:/etc/kernel/postinst.d/zz-update-grub: 2026-02-18T08:47:52.814 INFO:teuthology.orchestra.run.trial118.stdout:Sourcing file `/etc/default/grub' 2026-02-18T08:47:52.834 INFO:teuthology.orchestra.run.trial118.stdout:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-18T08:47:52.835 INFO:teuthology.orchestra.run.trial118.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-18T08:47:52.838 INFO:teuthology.orchestra.run.trial118.stdout:Generating grub configuration file ... 2026-02-18T08:47:52.919 INFO:teuthology.orchestra.run.trial118.stdout:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-18T08:47:52.926 INFO:teuthology.orchestra.run.trial118.stdout:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-18T08:47:53.100 INFO:teuthology.orchestra.run.trial118.stdout:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-18T08:47:53.105 INFO:teuthology.orchestra.run.trial118.stdout:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-18T08:47:53.175 INFO:teuthology.orchestra.run.trial118.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-18T08:47:53.176 INFO:teuthology.orchestra.run.trial118.stdout:Systems on them will not be added to the GRUB boot configuration. 2026-02-18T08:47:53.176 INFO:teuthology.orchestra.run.trial118.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-18T08:47:53.178 INFO:teuthology.orchestra.run.trial118.stdout:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-18T08:47:53.187 INFO:teuthology.orchestra.run.trial118.stdout:done 2026-02-18T08:47:53.611 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:47:53.611 INFO:teuthology.orchestra.run.trial118.stdout:Pending kernel upgrade! 2026-02-18T08:47:53.611 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:47:53.611 INFO:teuthology.orchestra.run.trial118.stdout:Running kernel version: 2026-02-18T08:47:53.611 INFO:teuthology.orchestra.run.trial118.stdout: 5.15.0-164-generic 2026-02-18T08:47:53.612 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:47:53.612 INFO:teuthology.orchestra.run.trial118.stdout:Diagnostics: 2026-02-18T08:47:53.612 INFO:teuthology.orchestra.run.trial118.stdout: The currently running kernel version is not the expected kernel version 5.15.0-170-generic. 2026-02-18T08:47:53.612 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:47:53.612 INFO:teuthology.orchestra.run.trial118.stdout:Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return] 2026-02-18T08:47:53.612 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:47:53.612 INFO:teuthology.orchestra.run.trial118.stdout:The processor microcode seems to be up-to-date. 2026-02-18T08:47:53.612 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:47:53.612 INFO:teuthology.orchestra.run.trial118.stdout:No services need to be restarted. 2026-02-18T08:47:53.612 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:47:53.612 INFO:teuthology.orchestra.run.trial118.stdout:No containers need to be restarted. 2026-02-18T08:47:53.613 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:47:53.613 INFO:teuthology.orchestra.run.trial118.stdout:No user sessions are running outdated binaries. 2026-02-18T08:47:53.613 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:47:53.613 INFO:teuthology.orchestra.run.trial118.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-18T08:47:54.268 DEBUG:teuthology.orchestra.run.trial118:> dpkg -s linux-image-generic 2026-02-18T08:47:54.319 INFO:teuthology.orchestra.run.trial118.stdout:Package: linux-image-generic 2026-02-18T08:47:54.319 INFO:teuthology.orchestra.run.trial118.stdout:Status: install ok installed 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout:Priority: optional 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout:Section: kernel 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout:Installed-Size: 21 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout:Maintainer: Ubuntu Kernel Team 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout:Architecture: amd64 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout:Source: linux-meta 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout:Version: 5.15.0.170.159 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout:Recommends: thermald 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout:Description: Generic Linux kernel image 2026-02-18T08:47:54.320 INFO:teuthology.orchestra.run.trial118.stdout: This package will always depend on the latest generic kernel image 2026-02-18T08:47:54.321 INFO:teuthology.orchestra.run.trial118.stdout: available. 2026-02-18T08:47:54.321 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-164-generic Expected: 5.15.0-170-generic 2026-02-18T08:47:54.321 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-18T08:47:54.321 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-18T08:47:54.321 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-18T08:47:54.321 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial118.front.sepia.ceph.com, path=None, version=distro) 2026-02-18T08:47:54.322 DEBUG:teuthology.orchestra.run.trial118:> sudo apt-get clean 2026-02-18T08:47:54.401 DEBUG:teuthology.orchestra.run.trial118:> sudo apt-get update 2026-02-18T08:47:54.843 INFO:teuthology.orchestra.run.trial118.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-18T08:47:54.911 INFO:teuthology.orchestra.run.trial118.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-18T08:47:54.982 INFO:teuthology.orchestra.run.trial118.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-18T08:47:55.054 INFO:teuthology.orchestra.run.trial118.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-18T08:47:55.722 INFO:teuthology.orchestra.run.trial118.stdout:Reading package lists... 2026-02-18T08:47:55.737 DEBUG:teuthology.orchestra.run.trial118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-18T08:47:55.770 INFO:teuthology.orchestra.run.trial118.stdout:Reading package lists... 2026-02-18T08:47:55.884 INFO:teuthology.orchestra.run.trial118.stdout:Building dependency tree... 2026-02-18T08:47:55.885 INFO:teuthology.orchestra.run.trial118.stdout:Reading state information... 2026-02-18T08:47:55.973 INFO:teuthology.orchestra.run.trial118.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-18T08:47:55.973 INFO:teuthology.orchestra.run.trial118.stdout:linux-image-generic set to manually installed. 2026-02-18T08:47:55.973 INFO:teuthology.orchestra.run.trial118.stdout:The following packages were automatically installed and are no longer required: 2026-02-18T08:47:55.974 INFO:teuthology.orchestra.run.trial118.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-18T08:47:55.974 INFO:teuthology.orchestra.run.trial118.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-18T08:47:55.974 INFO:teuthology.orchestra.run.trial118.stdout: sg3-utils-udev 2026-02-18T08:47:55.974 INFO:teuthology.orchestra.run.trial118.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-18T08:47:55.994 INFO:teuthology.orchestra.run.trial118.stdout:0 upgraded, 0 newly installed, 0 to remove and 53 not upgraded. 2026-02-18T08:47:55.996 DEBUG:teuthology.orchestra.run.trial118:> dpkg -s linux-image-generic 2026-02-18T08:47:56.047 INFO:teuthology.orchestra.run.trial118.stdout:Package: linux-image-generic 2026-02-18T08:47:56.047 INFO:teuthology.orchestra.run.trial118.stdout:Status: install ok installed 2026-02-18T08:47:56.047 INFO:teuthology.orchestra.run.trial118.stdout:Priority: optional 2026-02-18T08:47:56.047 INFO:teuthology.orchestra.run.trial118.stdout:Section: kernel 2026-02-18T08:47:56.047 INFO:teuthology.orchestra.run.trial118.stdout:Installed-Size: 21 2026-02-18T08:47:56.047 INFO:teuthology.orchestra.run.trial118.stdout:Maintainer: Ubuntu Kernel Team 2026-02-18T08:47:56.047 INFO:teuthology.orchestra.run.trial118.stdout:Architecture: amd64 2026-02-18T08:47:56.047 INFO:teuthology.orchestra.run.trial118.stdout:Source: linux-meta 2026-02-18T08:47:56.047 INFO:teuthology.orchestra.run.trial118.stdout:Version: 5.15.0.170.159 2026-02-18T08:47:56.047 INFO:teuthology.orchestra.run.trial118.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-18T08:47:56.048 INFO:teuthology.orchestra.run.trial118.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-18T08:47:56.048 INFO:teuthology.orchestra.run.trial118.stdout:Recommends: thermald 2026-02-18T08:47:56.048 INFO:teuthology.orchestra.run.trial118.stdout:Description: Generic Linux kernel image 2026-02-18T08:47:56.048 INFO:teuthology.orchestra.run.trial118.stdout: This package will always depend on the latest generic kernel image 2026-02-18T08:47:56.048 INFO:teuthology.orchestra.run.trial118.stdout: available. 2026-02-18T08:47:56.048 DEBUG:teuthology.orchestra.run.trial118:> mktemp 2026-02-18T08:47:56.094 INFO:teuthology.orchestra.run.trial118.stdout:/tmp/tmp.XwSrqBi2RF 2026-02-18T08:47:56.094 DEBUG:teuthology.orchestra.run.trial118:> sudo cp /boot/grub/grub.cfg /tmp/tmp.XwSrqBi2RF 2026-02-18T08:47:56.146 DEBUG:teuthology.orchestra.run.trial118:> sudo chmod 0666 /tmp/tmp.XwSrqBi2RF 2026-02-18T08:47:56.244 DEBUG:teuthology.orchestra.remote:trial118:/tmp/tmp.XwSrqBi2RF is 7KB 2026-02-18T08:47:56.252 DEBUG:teuthology.orchestra.run.trial118:> rm -fr /tmp/tmp.XwSrqBi2RF 2026-02-18T08:47:56.256 DEBUG:teuthology.orchestra.run.trial118:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2026-02-18T08:47:56.305 DEBUG:teuthology.orchestra.run.trial118:> set -ex 2026-02-18T08:47:56.305 DEBUG:teuthology.orchestra.run.trial118:> sudo dd of=/etc/grub.d/01_ceph_kernel 2026-02-18T08:47:56.306 DEBUG:teuthology.orchestra.run.trial118:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2026-02-18T08:47:56.363 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-170-generic 2026-02-18T08:47:56.364 DEBUG:teuthology.orchestra.run.trial118:> sudo update-grub 2026-02-18T08:47:56.608 INFO:teuthology.orchestra.run.trial118.stderr:Sourcing file `/etc/default/grub' 2026-02-18T08:47:56.627 INFO:teuthology.orchestra.run.trial118.stderr:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-18T08:47:56.628 INFO:teuthology.orchestra.run.trial118.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-18T08:47:56.630 INFO:teuthology.orchestra.run.trial118.stderr:Generating grub configuration file ... 2026-02-18T08:47:56.710 INFO:teuthology.orchestra.run.trial118.stderr:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-18T08:47:56.716 INFO:teuthology.orchestra.run.trial118.stderr:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-18T08:47:56.886 INFO:teuthology.orchestra.run.trial118.stderr:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-18T08:47:56.891 INFO:teuthology.orchestra.run.trial118.stderr:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-18T08:47:56.961 INFO:teuthology.orchestra.run.trial118.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-18T08:47:56.961 INFO:teuthology.orchestra.run.trial118.stderr:Systems on them will not be added to the GRUB boot configuration. 2026-02-18T08:47:56.961 INFO:teuthology.orchestra.run.trial118.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-18T08:47:56.964 INFO:teuthology.orchestra.run.trial118.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-18T08:47:56.971 INFO:teuthology.orchestra.run.trial118.stderr:done 2026-02-18T08:47:56.973 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial118 2026-02-18T08:47:56.973 DEBUG:teuthology.orchestra.run.trial118:> sudo bash -c ' 2026-02-18T08:47:56.974 DEBUG:teuthology.orchestra.run.trial118:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-18T08:47:56.974 DEBUG:teuthology.orchestra.run.trial118:> ' 2026-02-18T08:48:26.976 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-18T08:48:26.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial118.front.sepia.ceph.com' 2026-02-18T08:48:26.978 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-18T08:49:27.040 DEBUG:teuthology.orchestra.remote:timed out 2026-02-18T08:49:36.049 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial118.front.sepia.ceph.com' 2026-02-18T08:49:36.050 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-18T08:49:36.178 DEBUG:teuthology.orchestra.run.trial118:> true 2026-02-18T08:49:36.574 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial118.front.sepia.ceph.com' 2026-02-18T08:49:36.574 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-170-generic"... 2026-02-18T08:49:36.574 DEBUG:teuthology.orchestra.run.trial118:> uname -r 2026-02-18T08:49:36.621 INFO:teuthology.orchestra.run.trial118.stdout:5.15.0-170-generic 2026-02-18T08:49:36.621 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-170-generic vs 5.15.0-170-generic 2026-02-18T08:49:36.621 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-18T08:49:36.621 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2026-02-18T08:49:37.622 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-18T08:49:37.623 DEBUG:teuthology.orchestra.run.trial118:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-18T08:49:37.635 INFO:teuthology.orchestra.run.trial118.stdout:ttyS1 2026-02-18T08:49:37.659 DEBUG:teuthology.parallel:result is None 2026-02-18T08:49:37.659 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-18T08:49:37.667 INFO:teuthology.task.internal:Creating test directory... 2026-02-18T08:49:37.668 DEBUG:teuthology.orchestra.run.trial118:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-18T08:49:37.704 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-18T08:49:37.716 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-18T08:49:37.734 INFO:teuthology.task.internal:Creating archive directory... 2026-02-18T08:49:37.735 DEBUG:teuthology.orchestra.run.trial118:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-18T08:49:37.754 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-18T08:49:37.769 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-18T08:49:37.770 DEBUG:teuthology.orchestra.run.trial118:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-18T08:49:37.801 INFO:teuthology.orchestra.run.trial118.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:49:37.807 INFO:teuthology.orchestra.run.trial118.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:49:37.808 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-18T08:49:37.820 INFO:teuthology.task.internal:Configuring sudo... 2026-02-18T08:49:37.820 DEBUG:teuthology.orchestra.run.trial118:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-18T08:49:37.857 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-18T08:49:37.873 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-18T08:49:37.873 DEBUG:teuthology.orchestra.run.trial118:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-18T08:49:37.900 DEBUG:teuthology.orchestra.run.trial118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-18T08:49:37.944 DEBUG:teuthology.orchestra.run.trial118:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-18T08:49:37.988 DEBUG:teuthology.orchestra.run.trial118:> set -ex 2026-02-18T08:49:37.989 DEBUG:teuthology.orchestra.run.trial118:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-18T08:49:38.038 DEBUG:teuthology.orchestra.run.trial118:> sudo service rsyslog restart 2026-02-18T08:49:38.098 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-18T08:49:38.110 INFO:teuthology.task.internal:Starting timer... 2026-02-18T08:49:38.110 INFO:teuthology.run_tasks:Running task pcp... 2026-02-18T08:49:38.131 INFO:teuthology.run_tasks:Running task selinux... 2026-02-18T08:49:38.149 DEBUG:teuthology.task.selinux:Excluding trial118: OS 'ubuntu' does not support SELinux 2026-02-18T08:49:38.150 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-18T08:49:38.150 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-18T08:49:38.150 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-18T08:49:38.150 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-18T08:49:38.165 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-18T08:49:38.180 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-18T08:49:38.181 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial118.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-18T08:50:39.471 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial118.front.sepia.ceph.com')] 2026-02-18T08:50:39.474 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial118.front.sepia.ceph.com' 2026-02-18T08:50:39.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial118.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-18T08:50:39.586 DEBUG:teuthology.orchestra.run.trial118:> true 2026-02-18T08:50:39.831 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial118.front.sepia.ceph.com' 2026-02-18T08:50:39.832 INFO:teuthology.run_tasks:Running task clock... 2026-02-18T08:50:39.847 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-18T08:50:39.847 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-18T08:50:39.848 DEBUG:teuthology.orchestra.run.trial118:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-18T08:50:39.891 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-18T08:50:39.892 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: Command line: ntpd -gq 2026-02-18T08:50:39.892 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: ---------------------------------------------------- 2026-02-18T08:50:39.892 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: ntp-4 is maintained by Network Time Foundation, 2026-02-18T08:50:39.892 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-18T08:50:39.892 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: corporation. Support and training for ntp-4 are 2026-02-18T08:50:39.892 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: available at https://www.nwtime.org/support 2026-02-18T08:50:39.892 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: ---------------------------------------------------- 2026-02-18T08:50:39.892 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: proto: precision = 0.020 usec (-25) 2026-02-18T08:50:39.892 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: basedate set to 2022-02-04 2026-02-18T08:50:39.892 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: gps base set to 2022-02-06 (week 2196) 2026-02-18T08:50:39.893 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-18T08:50:39.893 INFO:teuthology.orchestra.run.trial118.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-18T08:50:39.893 INFO:teuthology.orchestra.run.trial118.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-18T08:50:39.894 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: restrict ::: KOD does nothing without LIMITED. 2026-02-18T08:50:39.894 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: Listen and drop on 0 v6wildcard [::]:123 2026-02-18T08:50:39.894 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-18T08:50:39.895 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: Listen normally on 2 lo 127.0.0.1:123 2026-02-18T08:50:39.895 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: Listen normally on 3 enp1s0f0 10.20.193.118:123 2026-02-18T08:50:39.895 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: Listen normally on 4 lo [::1]:123 2026-02-18T08:50:39.895 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:6138%2]:123 2026-02-18T08:50:39.895 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:40 ntpd[4912]: Listening on routing socket on fd #22 for interface updates 2026-02-18T08:50:40.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:41 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:40.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:41 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:40.957 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:41 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:40.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:41 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:40.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:41 ntpd[4912]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-18T08:50:40.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:41 ntpd[4912]: can't open /var/log/ntpstats/peerstats.20260218: Permission denied 2026-02-18T08:50:41.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:41.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:41.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:41.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:41.899 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:41.899 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:41.899 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-18T08:50:41.899 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: can't open /var/log/ntpstats/peerstats.20260218: Permission denied 2026-02-18T08:50:41.904 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:41.904 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:41.904 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-18T08:50:41.904 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:42 ntpd[4912]: can't open /var/log/ntpstats/peerstats.20260218: Permission denied 2026-02-18T08:50:42.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:43 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:42.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:43 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:42.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:43 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:42.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:43 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:42.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:43 ntpd[4912]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-18T08:50:42.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:43 ntpd[4912]: can't open /var/log/ntpstats/peerstats.20260218: Permission denied 2026-02-18T08:50:43.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:43.892 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:43.892 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:43.892 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:43.899 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:43.899 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:43.899 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-18T08:50:43.899 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: can't open /var/log/ntpstats/peerstats.20260218: Permission denied 2026-02-18T08:50:43.905 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:43.905 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:43.905 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-18T08:50:43.905 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:44 ntpd[4912]: can't open /var/log/ntpstats/peerstats.20260218: Permission denied 2026-02-18T08:50:44.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:44.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:44.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:44.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:44.894 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:44.894 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:44.894 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-18T08:50:44.894 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: can't open /var/log/ntpstats/peerstats.20260218: Permission denied 2026-02-18T08:50:44.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:44.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:44.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-18T08:50:44.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:45 ntpd[4912]: can't open /var/log/ntpstats/peerstats.20260218: Permission denied 2026-02-18T08:50:45.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:46 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:45.892 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:46 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:45.892 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:46 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:45.892 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:46 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:45.898 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:46 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:45.899 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:46 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:45.899 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:46 ntpd[4912]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-18T08:50:45.899 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:46 ntpd[4912]: can't open /var/log/ntpstats/peerstats.20260218: Permission denied 2026-02-18T08:50:46.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:46.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:46.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:46.891 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:46.894 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:46.894 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:46.894 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-18T08:50:46.894 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: can't open /var/log/ntpstats/peerstats.20260218: Permission denied 2026-02-18T08:50:46.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-18T08:50:46.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: can't open /var/log/ntpstats/rawstats.20260218: Permission denied 2026-02-18T08:50:46.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-18T08:50:46.958 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:47 ntpd[4912]: can't open /var/log/ntpstats/peerstats.20260218: Permission denied 2026-02-18T08:50:46.959 INFO:teuthology.orchestra.run.trial118.stdout:18 Feb 08:50:46 ntpd[4912]: ntpd: time set -0.256860 s 2026-02-18T08:50:46.959 INFO:teuthology.orchestra.run.trial118.stdout:ntpd: time set -0.256860s 2026-02-18T08:50:46.959 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:46 ntpd[4912]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-18T08:50:46.959 INFO:teuthology.orchestra.run.trial118.stderr:18 Feb 08:50:46 ntpd[4912]: can't open /var/log/ntpstats/loopstats.20260218: Permission denied 2026-02-18T08:50:47.012 INFO:teuthology.orchestra.run.trial118.stdout: remote refid st t when poll reach delay offset jitter 2026-02-18T08:50:47.012 INFO:teuthology.orchestra.run.trial118.stdout:============================================================================== 2026-02-18T08:50:47.012 INFO:teuthology.orchestra.run.trial118.stdout: 172-234-37-140. .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-18T08:50:47.012 INFO:teuthology.orchestra.run.trial118.stdout: pool-96-231-54- .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-18T08:50:47.012 INFO:teuthology.orchestra.run.trial118.stdout: nyc2.us.ntp.li .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-18T08:50:47.012 INFO:teuthology.orchestra.run.trial118.stdout: 198.137.202.32 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-18T08:50:47.014 INFO:teuthology.run_tasks:Running task install... 2026-02-18T08:50:47.028 DEBUG:teuthology.task.install:project ceph 2026-02-18T08:50:47.029 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '325850157d309f65985eea9ec9bb9f1d56726fee'}} 2026-02-18T08:50:47.029 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '325850157d309f65985eea9ec9bb9f1d56726fee'} 2026-02-18T08:50:47.029 INFO:teuthology.task.install:Using flavor: default 2026-02-18T08:50:47.039 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-18T08:50:47.039 INFO:teuthology.task.install:extra packages: [] 2026-02-18T08:50:47.040 DEBUG:teuthology.orchestra.run.trial118:> sudo apt-key list | grep Ceph 2026-02-18T08:50:47.103 INFO:teuthology.orchestra.run.trial118.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-18T08:50:47.130 INFO:teuthology.orchestra.run.trial118.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-18T08:50:47.130 INFO:teuthology.orchestra.run.trial118.stdout:uid [ unknown] Ceph.com (release key) 2026-02-18T08:50:47.132 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-18T08:50:47.132 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=325850157d309f65985eea9ec9bb9f1d56726fee 2026-02-18T08:50:47.313 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default/ 2026-02-18T08:50:47.313 INFO:teuthology.task.install.deb:Package version is 20.3.0-5271-g32585015-1jammy 2026-02-18T08:50:47.464 DEBUG:teuthology.orchestra.run.trial118:> set -ex 2026-02-18T08:50:47.464 DEBUG:teuthology.orchestra.run.trial118:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-18T08:50:47.474 DEBUG:teuthology.orchestra.run.trial118:> sudo apt-get update 2026-02-18T08:50:47.629 INFO:teuthology.orchestra.run.trial118.stdout:Ign:1 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy InRelease 2026-02-18T08:50:47.640 INFO:teuthology.orchestra.run.trial118.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-18T08:50:47.652 INFO:teuthology.orchestra.run.trial118.stdout:Ign:3 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-18T08:50:47.664 INFO:teuthology.orchestra.run.trial118.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-18T08:50:48.169 INFO:teuthology.orchestra.run.trial118.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-18T08:50:48.237 INFO:teuthology.orchestra.run.trial118.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-18T08:50:48.309 INFO:teuthology.orchestra.run.trial118.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-18T08:50:48.381 INFO:teuthology.orchestra.run.trial118.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-18T08:50:48.524 INFO:teuthology.orchestra.run.trial118.stdout:Fetched 26.9 kB in 1s (30.3 kB/s) 2026-02-18T08:50:49.046 INFO:teuthology.orchestra.run.trial118.stdout:Reading package lists... 2026-02-18T08:50:49.060 DEBUG:teuthology.orchestra.run.trial118:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-5271-g32585015-1jammy cephadm=20.3.0-5271-g32585015-1jammy ceph-mds=20.3.0-5271-g32585015-1jammy ceph-mgr=20.3.0-5271-g32585015-1jammy ceph-osd=20.3.0-5271-g32585015-1jammy ceph-osd-classic=20.3.0-5271-g32585015-1jammy ceph-common=20.3.0-5271-g32585015-1jammy ceph-fuse=20.3.0-5271-g32585015-1jammy ceph-test=20.3.0-5271-g32585015-1jammy ceph-volume=20.3.0-5271-g32585015-1jammy radosgw=20.3.0-5271-g32585015-1jammy python3-rados=20.3.0-5271-g32585015-1jammy python3-rgw=20.3.0-5271-g32585015-1jammy python3-cephfs=20.3.0-5271-g32585015-1jammy python3-rbd=20.3.0-5271-g32585015-1jammy libcephfs2=20.3.0-5271-g32585015-1jammy libcephfs-dev=20.3.0-5271-g32585015-1jammy librados2=20.3.0-5271-g32585015-1jammy librbd1=20.3.0-5271-g32585015-1jammy rbd-fuse=20.3.0-5271-g32585015-1jammy 2026-02-18T08:50:49.094 INFO:teuthology.orchestra.run.trial118.stdout:Reading package lists... 2026-02-18T08:50:49.201 INFO:teuthology.orchestra.run.trial118.stdout:Building dependency tree... 2026-02-18T08:50:49.201 INFO:teuthology.orchestra.run.trial118.stdout:Reading state information... 2026-02-18T08:50:49.290 INFO:teuthology.orchestra.run.trial118.stdout:The following packages were automatically installed and are no longer required: 2026-02-18T08:50:49.291 INFO:teuthology.orchestra.run.trial118.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-18T08:50:49.291 INFO:teuthology.orchestra.run.trial118.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-18T08:50:49.291 INFO:teuthology.orchestra.run.trial118.stdout: sg3-utils-udev 2026-02-18T08:50:49.291 INFO:teuthology.orchestra.run.trial118.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-18T08:50:49.291 INFO:teuthology.orchestra.run.trial118.stdout:The following additional packages will be installed: 2026-02-18T08:50:49.291 INFO:teuthology.orchestra.run.trial118.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-18T08:50:49.291 INFO:teuthology.orchestra.run.trial118.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-18T08:50:49.291 INFO:teuthology.orchestra.run.trial118.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-18T08:50:49.291 INFO:teuthology.orchestra.run.trial118.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-18T08:50:49.292 INFO:teuthology.orchestra.run.trial118.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-18T08:50:49.293 INFO:teuthology.orchestra.run.trial118.stdout:Suggested packages: 2026-02-18T08:50:49.293 INFO:teuthology.orchestra.run.trial118.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-18T08:50:49.293 INFO:teuthology.orchestra.run.trial118.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-18T08:50:49.293 INFO:teuthology.orchestra.run.trial118.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-18T08:50:49.293 INFO:teuthology.orchestra.run.trial118.stdout:Recommended packages: 2026-02-18T08:50:49.293 INFO:teuthology.orchestra.run.trial118.stdout: btrfs-tools 2026-02-18T08:50:49.325 INFO:teuthology.orchestra.run.trial118.stdout:The following NEW packages will be installed: 2026-02-18T08:50:49.325 INFO:teuthology.orchestra.run.trial118.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-18T08:50:49.325 INFO:teuthology.orchestra.run.trial118.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-18T08:50:49.325 INFO:teuthology.orchestra.run.trial118.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-18T08:50:49.325 INFO:teuthology.orchestra.run.trial118.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-18T08:50:49.325 INFO:teuthology.orchestra.run.trial118.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-18T08:50:49.325 INFO:teuthology.orchestra.run.trial118.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-18T08:50:49.326 INFO:teuthology.orchestra.run.trial118.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-18T08:50:49.326 INFO:teuthology.orchestra.run.trial118.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-18T08:50:49.326 INFO:teuthology.orchestra.run.trial118.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-18T08:50:49.326 INFO:teuthology.orchestra.run.trial118.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-18T08:50:49.326 INFO:teuthology.orchestra.run.trial118.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-18T08:50:49.326 INFO:teuthology.orchestra.run.trial118.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-18T08:50:49.326 INFO:teuthology.orchestra.run.trial118.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-18T08:50:49.326 INFO:teuthology.orchestra.run.trial118.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-18T08:50:49.326 INFO:teuthology.orchestra.run.trial118.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-18T08:50:49.326 INFO:teuthology.orchestra.run.trial118.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-18T08:50:49.326 INFO:teuthology.orchestra.run.trial118.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-18T08:50:49.327 INFO:teuthology.orchestra.run.trial118.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-18T08:50:49.327 INFO:teuthology.orchestra.run.trial118.stdout: socat xmlstarlet 2026-02-18T08:50:49.401 INFO:teuthology.orchestra.run.trial118.stdout:0 upgraded, 87 newly installed, 0 to remove and 53 not upgraded. 2026-02-18T08:50:49.401 INFO:teuthology.orchestra.run.trial118.stdout:Need to get 702 MB of archives. 2026-02-18T08:50:49.401 INFO:teuthology.orchestra.run.trial118.stdout:After this operation, 2670 MB of additional disk space will be used. 2026-02-18T08:50:49.402 INFO:teuthology.orchestra.run.trial118.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-5271-g32585015-1jammy [3799 kB] 2026-02-18T08:50:49.507 INFO:teuthology.orchestra.run.trial118.stdout:Get:2 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-5271-g32585015-1jammy [2910 kB] 2026-02-18T08:50:49.545 INFO:teuthology.orchestra.run.trial118.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-5271-g32585015-1jammy [882 kB] 2026-02-18T08:50:49.560 INFO:teuthology.orchestra.run.trial118.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-5271-g32585015-1jammy [359 kB] 2026-02-18T08:50:49.565 INFO:teuthology.orchestra.run.trial118.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-5271-g32585015-1jammy [32.8 kB] 2026-02-18T08:50:49.566 INFO:teuthology.orchestra.run.trial118.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-5271-g32585015-1jammy [225 kB] 2026-02-18T08:50:49.569 INFO:teuthology.orchestra.run.trial118.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-5271-g32585015-1jammy [88.4 kB] 2026-02-18T08:50:49.571 INFO:teuthology.orchestra.run.trial118.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-5271-g32585015-1jammy [339 kB] 2026-02-18T08:50:49.576 INFO:teuthology.orchestra.run.trial118.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-5271-g32585015-1jammy [9091 kB] 2026-02-18T08:50:49.677 INFO:teuthology.orchestra.run.trial118.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-18T08:50:49.726 INFO:teuthology.orchestra.run.trial118.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-5271-g32585015-1jammy [112 kB] 2026-02-18T08:50:49.727 INFO:teuthology.orchestra.run.trial118.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-5271-g32585015-1jammy [261 kB] 2026-02-18T08:50:49.731 INFO:teuthology.orchestra.run.trial118.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-5271-g32585015-1jammy [31.5 MB] 2026-02-18T08:50:50.015 INFO:teuthology.orchestra.run.trial118.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-18T08:50:50.025 INFO:teuthology.orchestra.run.trial118.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-18T08:50:50.095 INFO:teuthology.orchestra.run.trial118.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-18T08:50:50.253 INFO:teuthology.orchestra.run.trial118.stdout:Get:17 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-5271-g32585015-1jammy [4983 kB] 2026-02-18T08:50:50.298 INFO:teuthology.orchestra.run.trial118.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-18T08:50:50.307 INFO:teuthology.orchestra.run.trial118.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-18T08:50:50.330 INFO:teuthology.orchestra.run.trial118.stdout:Get:20 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-5271-g32585015-1jammy [293 kB] 2026-02-18T08:50:50.334 INFO:teuthology.orchestra.run.trial118.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-18T08:50:50.335 INFO:teuthology.orchestra.run.trial118.stdout:Get:22 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-5271-g32585015-1jammy [127 kB] 2026-02-18T08:50:50.336 INFO:teuthology.orchestra.run.trial118.stdout:Get:23 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-5271-g32585015-1jammy [922 kB] 2026-02-18T08:50:50.342 INFO:teuthology.orchestra.run.trial118.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-18T08:50:50.344 INFO:teuthology.orchestra.run.trial118.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-18T08:50:50.345 INFO:teuthology.orchestra.run.trial118.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-18T08:50:50.346 INFO:teuthology.orchestra.run.trial118.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-18T08:50:50.351 INFO:teuthology.orchestra.run.trial118.stdout:Get:28 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-5271-g32585015-1jammy [7537 kB] 2026-02-18T08:50:50.363 INFO:teuthology.orchestra.run.trial118.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-18T08:50:50.364 INFO:teuthology.orchestra.run.trial118.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-18T08:50:50.369 INFO:teuthology.orchestra.run.trial118.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-18T08:50:50.436 INFO:teuthology.orchestra.run.trial118.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-18T08:50:50.437 INFO:teuthology.orchestra.run.trial118.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-18T08:50:50.437 INFO:teuthology.orchestra.run.trial118.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-18T08:50:50.437 INFO:teuthology.orchestra.run.trial118.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-18T08:50:50.438 INFO:teuthology.orchestra.run.trial118.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-18T08:50:50.438 INFO:teuthology.orchestra.run.trial118.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-18T08:50:50.438 INFO:teuthology.orchestra.run.trial118.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-18T08:50:50.442 INFO:teuthology.orchestra.run.trial118.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-18T08:50:50.442 INFO:teuthology.orchestra.run.trial118.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-18T08:50:50.476 INFO:teuthology.orchestra.run.trial118.stdout:Get:41 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5271-g32585015-1jammy [468 MB] 2026-02-18T08:50:50.512 INFO:teuthology.orchestra.run.trial118.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-18T08:50:50.515 INFO:teuthology.orchestra.run.trial118.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-18T08:50:50.515 INFO:teuthology.orchestra.run.trial118.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-18T08:50:50.516 INFO:teuthology.orchestra.run.trial118.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-18T08:50:50.541 INFO:teuthology.orchestra.run.trial118.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-18T08:50:50.543 INFO:teuthology.orchestra.run.trial118.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-18T08:50:50.543 INFO:teuthology.orchestra.run.trial118.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-18T08:50:50.601 INFO:teuthology.orchestra.run.trial118.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-18T08:50:50.601 INFO:teuthology.orchestra.run.trial118.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-18T08:50:50.602 INFO:teuthology.orchestra.run.trial118.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-18T08:50:50.602 INFO:teuthology.orchestra.run.trial118.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-18T08:50:50.603 INFO:teuthology.orchestra.run.trial118.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-18T08:50:50.660 INFO:teuthology.orchestra.run.trial118.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-18T08:50:50.663 INFO:teuthology.orchestra.run.trial118.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-18T08:50:50.665 INFO:teuthology.orchestra.run.trial118.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-18T08:50:50.666 INFO:teuthology.orchestra.run.trial118.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-18T08:50:50.733 INFO:teuthology.orchestra.run.trial118.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-18T08:50:50.736 INFO:teuthology.orchestra.run.trial118.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-18T08:50:50.738 INFO:teuthology.orchestra.run.trial118.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-18T08:50:50.742 INFO:teuthology.orchestra.run.trial118.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-18T08:50:50.744 INFO:teuthology.orchestra.run.trial118.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-18T08:50:50.745 INFO:teuthology.orchestra.run.trial118.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-18T08:50:50.805 INFO:teuthology.orchestra.run.trial118.stdout:Get:64 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-18T08:50:50.806 INFO:teuthology.orchestra.run.trial118.stdout:Get:65 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-18T08:50:50.807 INFO:teuthology.orchestra.run.trial118.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-18T08:50:50.813 INFO:teuthology.orchestra.run.trial118.stdout:Get:67 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-18T08:50:50.877 INFO:teuthology.orchestra.run.trial118.stdout:Get:68 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-18T08:50:50.879 INFO:teuthology.orchestra.run.trial118.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-18T08:50:50.879 INFO:teuthology.orchestra.run.trial118.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-18T08:50:50.880 INFO:teuthology.orchestra.run.trial118.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-18T08:50:58.412 INFO:teuthology.orchestra.run.trial118.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-5271-g32585015-1jammy [180 kB] 2026-02-18T08:50:58.413 INFO:teuthology.orchestra.run.trial118.stdout:Get:73 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-5271-g32585015-1jammy [14.1 kB] 2026-02-18T08:50:58.413 INFO:teuthology.orchestra.run.trial118.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-5271-g32585015-1jammy [1030 kB] 2026-02-18T08:50:58.418 INFO:teuthology.orchestra.run.trial118.stdout:Get:75 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-5271-g32585015-1jammy [2437 kB] 2026-02-18T08:50:58.431 INFO:teuthology.orchestra.run.trial118.stdout:Get:76 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-5271-g32585015-1jammy [1069 kB] 2026-02-18T08:50:58.437 INFO:teuthology.orchestra.run.trial118.stdout:Get:77 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-5271-g32585015-1jammy [190 kB] 2026-02-18T08:50:58.438 INFO:teuthology.orchestra.run.trial118.stdout:Get:78 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-5271-g32585015-1jammy [54.7 MB] 2026-02-18T08:50:59.336 INFO:teuthology.orchestra.run.trial118.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-5271-g32585015-1jammy [8626 kB] 2026-02-18T08:50:59.463 INFO:teuthology.orchestra.run.trial118.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-5271-g32585015-1jammy [14.2 kB] 2026-02-18T08:50:59.463 INFO:teuthology.orchestra.run.trial118.stdout:Get:81 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-5271-g32585015-1jammy [68.6 MB] 2026-02-18T08:51:00.610 INFO:teuthology.orchestra.run.trial118.stdout:Get:82 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-5271-g32585015-1jammy [136 kB] 2026-02-18T08:51:00.611 INFO:teuthology.orchestra.run.trial118.stdout:Get:83 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-5271-g32585015-1jammy [45.1 kB] 2026-02-18T08:51:00.612 INFO:teuthology.orchestra.run.trial118.stdout:Get:84 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-5271-g32585015-1jammy [31.7 kB] 2026-02-18T08:51:00.612 INFO:teuthology.orchestra.run.trial118.stdout:Get:85 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-5271-g32585015-1jammy [44.2 kB] 2026-02-18T08:51:00.612 INFO:teuthology.orchestra.run.trial118.stdout:Get:86 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-5271-g32585015-1jammy [18.3 MB] 2026-02-18T08:51:00.892 INFO:teuthology.orchestra.run.trial118.stdout:Get:87 https://1.chacra.ceph.com/r/ceph/wip-bharath18-testing-2026-02-13-0856/325850157d309f65985eea9ec9bb9f1d56726fee/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-5271-g32585015-1jammy [93.7 kB] 2026-02-18T08:51:01.166 INFO:teuthology.orchestra.run.trial118.stdout:Fetched 702 MB in 12s (60.7 MB/s) 2026-02-18T08:51:01.182 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-18T08:51:01.338 INFO:teuthology.orchestra.run.trial118.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167438 files and directories currently installed.) 2026-02-18T08:51:01.340 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-18T08:51:01.340 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-18T08:51:01.361 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-18T08:51:01.373 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-18T08:51:01.373 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-18T08:51:01.389 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-18T08:51:01.402 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-18T08:51:01.402 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-18T08:51:01.422 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-18T08:51:01.430 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-18T08:51:01.436 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-18T08:51:01.470 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-18T08:51:01.478 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-18T08:51:01.479 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-18T08:51:01.495 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-18T08:51:01.503 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-18T08:51:01.503 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-18T08:51:01.525 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-18T08:51:01.532 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-18T08:51:01.533 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-18T08:51:01.549 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package librados2. 2026-02-18T08:51:01.557 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../07-librados2_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:01.558 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking librados2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:01.601 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libnbd0. 2026-02-18T08:51:01.610 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-18T08:51:01.610 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-18T08:51:01.624 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package librbd1. 2026-02-18T08:51:01.632 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../09-librbd1_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:01.633 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking librbd1 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:01.671 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libcephfs2. 2026-02-18T08:51:01.679 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:01.680 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libcephfs2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:01.699 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-rados. 2026-02-18T08:51:01.707 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../11-python3-rados_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:01.708 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-rados (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:01.725 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-18T08:51:01.733 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-18T08:51:01.734 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-ceph-argparse (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:01.747 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-cephfs. 2026-02-18T08:51:01.756 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:01.756 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-cephfs (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:01.771 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-18T08:51:01.780 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-18T08:51:01.780 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-ceph-common (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:01.798 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-18T08:51:01.806 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-18T08:51:01.806 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-18T08:51:01.823 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-prettytable. 2026-02-18T08:51:01.831 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-18T08:51:01.832 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-18T08:51:01.846 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-rbd. 2026-02-18T08:51:01.854 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:01.854 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-rbd (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:01.871 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-18T08:51:01.879 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-18T08:51:01.880 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-18T08:51:01.898 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package librgw2. 2026-02-18T08:51:01.906 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../19-librgw2_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:01.907 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking librgw2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:02.011 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-rgw. 2026-02-18T08:51:02.019 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:02.020 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-rgw (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:02.035 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-18T08:51:02.043 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-18T08:51:02.043 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-18T08:51:02.057 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libradosstriper1. 2026-02-18T08:51:02.066 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:02.067 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libradosstriper1 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:02.083 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-common. 2026-02-18T08:51:02.091 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../23-ceph-common_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:02.092 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-common (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:02.357 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-base. 2026-02-18T08:51:02.366 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../24-ceph-base_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:02.371 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-base (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:02.428 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-18T08:51:02.436 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-18T08:51:02.436 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-18T08:51:02.451 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-cheroot. 2026-02-18T08:51:02.460 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-18T08:51:02.460 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-18T08:51:02.477 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-18T08:51:02.485 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-18T08:51:02.486 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-18T08:51:02.500 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-18T08:51:02.508 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-18T08:51:02.509 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-18T08:51:02.524 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-18T08:51:02.532 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-18T08:51:02.532 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-18T08:51:02.547 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-tempora. 2026-02-18T08:51:02.556 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-18T08:51:02.556 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-18T08:51:02.570 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-portend. 2026-02-18T08:51:02.579 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-18T08:51:02.579 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-18T08:51:02.593 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-18T08:51:02.602 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-18T08:51:02.602 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-18T08:51:02.617 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-18T08:51:02.625 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-18T08:51:02.626 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-18T08:51:02.650 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-natsort. 2026-02-18T08:51:02.659 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-18T08:51:02.659 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-18T08:51:02.675 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-18T08:51:02.683 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-18T08:51:02.684 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-mgr-modules-core (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:02.715 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-18T08:51:02.724 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:02.724 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:02.739 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-mgr. 2026-02-18T08:51:02.747 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:02.747 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-mgr (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:02.769 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-mon. 2026-02-18T08:51:02.777 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:02.777 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-mon (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:02.843 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-18T08:51:02.851 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-18T08:51:02.852 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-18T08:51:02.867 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-18T08:51:02.876 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:02.876 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-osd-classic (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:06.897 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-osd. 2026-02-18T08:51:06.910 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:06.910 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-osd (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:06.930 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph. 2026-02-18T08:51:06.942 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../42-ceph_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:06.942 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:06.959 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-fuse. 2026-02-18T08:51:06.972 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:06.972 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-fuse (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:06.995 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-mds. 2026-02-18T08:51:07.005 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:07.006 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-mds (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:07.039 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package cephadm. 2026-02-18T08:51:07.047 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../45-cephadm_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:07.047 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking cephadm (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:07.063 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-18T08:51:07.072 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-18T08:51:07.072 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-18T08:51:07.093 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-18T08:51:07.101 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-18T08:51:07.102 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-mgr-cephadm (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:07.123 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-18T08:51:07.131 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-18T08:51:07.132 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-18T08:51:07.147 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-routes. 2026-02-18T08:51:07.155 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-18T08:51:07.155 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-18T08:51:07.175 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-18T08:51:07.183 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-18T08:51:07.184 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-mgr-dashboard (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:07.863 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-18T08:51:07.872 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-18T08:51:07.872 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-18T08:51:07.911 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-joblib. 2026-02-18T08:51:07.920 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-18T08:51:07.920 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-18T08:51:07.947 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-18T08:51:07.955 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-18T08:51:07.955 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-18T08:51:07.970 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-sklearn. 2026-02-18T08:51:07.979 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-18T08:51:07.980 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-18T08:51:08.063 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-18T08:51:08.072 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-18T08:51:08.072 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:08.232 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-cachetools. 2026-02-18T08:51:08.241 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-18T08:51:08.241 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-18T08:51:08.256 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-rsa. 2026-02-18T08:51:08.264 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-18T08:51:08.265 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-18T08:51:08.283 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-google-auth. 2026-02-18T08:51:08.292 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-18T08:51:08.292 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-18T08:51:08.309 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-18T08:51:08.318 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-18T08:51:08.318 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-18T08:51:08.334 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-websocket. 2026-02-18T08:51:08.343 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-18T08:51:08.343 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-18T08:51:08.360 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-18T08:51:08.369 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-18T08:51:08.369 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-18T08:51:08.464 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-18T08:51:08.473 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-18T08:51:08.473 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:08.488 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-18T08:51:08.496 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-18T08:51:08.497 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-18T08:51:08.512 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-18T08:51:08.521 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-18T08:51:08.521 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-18T08:51:08.536 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package jq. 2026-02-18T08:51:08.544 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-18T08:51:08.545 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-18T08:51:08.559 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package socat. 2026-02-18T08:51:08.567 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-18T08:51:08.568 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-18T08:51:08.587 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package xmlstarlet. 2026-02-18T08:51:08.596 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-18T08:51:08.596 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-18T08:51:08.632 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-test. 2026-02-18T08:51:08.640 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../68-ceph-test_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:08.641 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-test (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:09.293 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package ceph-volume. 2026-02-18T08:51:09.301 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-5271-g32585015-1jammy_all.deb ... 2026-02-18T08:51:09.302 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking ceph-volume (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:09.325 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-18T08:51:09.333 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:09.334 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libcephfs-daemon (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:09.348 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-18T08:51:09.357 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:09.357 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libcephfs-proxy2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:09.370 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-18T08:51:09.379 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:09.379 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking libcephfs-dev (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:09.395 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package nvme-cli. 2026-02-18T08:51:09.404 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-18T08:51:09.404 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-18T08:51:09.436 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-18T08:51:09.445 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-18T08:51:09.445 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-18T08:51:09.475 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-18T08:51:09.484 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-18T08:51:09.484 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-18T08:51:09.499 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-pluggy. 2026-02-18T08:51:09.507 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-18T08:51:09.507 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-18T08:51:09.523 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-psutil. 2026-02-18T08:51:09.532 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-18T08:51:09.532 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-18T08:51:09.550 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-py. 2026-02-18T08:51:09.559 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-18T08:51:09.559 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-18T08:51:09.579 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-pygments. 2026-02-18T08:51:09.587 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-18T08:51:09.587 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-18T08:51:09.629 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-toml. 2026-02-18T08:51:09.638 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-18T08:51:09.638 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-18T08:51:09.653 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-pytest. 2026-02-18T08:51:09.661 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-18T08:51:09.662 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-18T08:51:09.684 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-simplejson. 2026-02-18T08:51:09.692 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-18T08:51:09.693 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-18T08:51:09.718 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package python3-webob. 2026-02-18T08:51:09.726 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-18T08:51:09.727 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-18T08:51:09.743 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-18T08:51:09.752 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-18T08:51:09.752 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-18T08:51:09.819 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package radosgw. 2026-02-18T08:51:09.828 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../85-radosgw_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:09.828 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking radosgw (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:09.986 INFO:teuthology.orchestra.run.trial118.stdout:Selecting previously unselected package rbd-fuse. 2026-02-18T08:51:09.995 INFO:teuthology.orchestra.run.trial118.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-5271-g32585015-1jammy_amd64.deb ... 2026-02-18T08:51:09.996 INFO:teuthology.orchestra.run.trial118.stdout:Unpacking rbd-fuse (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:10.023 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-18T08:51:10.089 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-18T08:51:10.090 INFO:teuthology.orchestra.run.trial118.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-18T08:51:10.162 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-18T08:51:10.453 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-18T08:51:10.860 INFO:teuthology.orchestra.run.trial118.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-18T08:51:10.870 INFO:teuthology.orchestra.run.trial118.stdout:Setting up cephadm (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:10.912 INFO:teuthology.orchestra.run.trial118.stdout:Adding system user cephadm....done 2026-02-18T08:51:10.919 INFO:teuthology.orchestra.run.trial118.stdout:usermod: no changes 2026-02-18T08:51:10.925 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-18T08:51:10.995 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-18T08:51:10.996 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-18T08:51:11.067 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-18T08:51:11.142 INFO:teuthology.orchestra.run.trial118.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-18T08:51:11.143 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-18T08:51:11.241 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-18T08:51:11.368 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-18T08:51:11.441 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-18T08:51:11.517 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-ceph-argparse (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:11.581 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-18T08:51:11.582 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-18T08:51:11.583 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-18T08:51:11.584 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-18T08:51:11.584 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-18T08:51:11.706 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-18T08:51:11.785 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libcephfs-proxy2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:11.785 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-18T08:51:11.864 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-18T08:51:11.951 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-18T08:51:12.201 INFO:teuthology.orchestra.run.trial118.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-18T08:51:12.202 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-18T08:51:12.297 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-18T08:51:12.425 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-18T08:51:12.508 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-18T08:51:12.579 INFO:teuthology.orchestra.run.trial118.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-18T08:51:12.580 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-ceph-common (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:12.670 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-18T08:51:13.181 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-18T08:51:13.186 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-18T08:51:13.264 INFO:teuthology.orchestra.run.trial118.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-18T08:51:13.264 INFO:teuthology.orchestra.run.trial118.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-18T08:51:13.265 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-18T08:51:13.338 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-18T08:51:13.411 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-18T08:51:13.412 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-18T08:51:13.488 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-18T08:51:13.561 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-18T08:51:13.637 INFO:teuthology.orchestra.run.trial118.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-18T08:51:13.638 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-18T08:51:13.722 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-18T08:51:13.723 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-18T08:51:13.799 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-18T08:51:13.884 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-18T08:51:13.960 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-18T08:51:13.961 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-18T08:51:14.094 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-18T08:51:14.166 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-18T08:51:14.167 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-18T08:51:14.249 INFO:teuthology.orchestra.run.trial118.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-18T08:51:14.250 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-18T08:51:14.383 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-18T08:51:14.384 INFO:teuthology.orchestra.run.trial118.stdout:Setting up librados2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:14.385 INFO:teuthology.orchestra.run.trial118.stdout:Setting up librgw2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:14.385 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libsqlite3-mod-ceph (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:14.386 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-18T08:51:14.919 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libcephfs2 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:14.920 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libradosstriper1 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:14.921 INFO:teuthology.orchestra.run.trial118.stdout:Setting up librbd1 (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:14.922 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-mgr-modules-core (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:14.923 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-fuse (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:14.994 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-18T08:51:14.994 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-18T08:51:15.404 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libcephfs-dev (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:15.405 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-rados (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:15.406 INFO:teuthology.orchestra.run.trial118.stdout:Setting up libcephfs-daemon (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:15.407 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-rbd (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:15.408 INFO:teuthology.orchestra.run.trial118.stdout:Setting up rbd-fuse (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:15.409 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-rgw (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:15.409 INFO:teuthology.orchestra.run.trial118.stdout:Setting up python3-cephfs (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:15.410 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-common (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:15.437 INFO:teuthology.orchestra.run.trial118.stdout:Adding group ceph....done 2026-02-18T08:51:15.472 INFO:teuthology.orchestra.run.trial118.stdout:Adding system user ceph....done 2026-02-18T08:51:15.482 INFO:teuthology.orchestra.run.trial118.stdout:Setting system user ceph properties....done 2026-02-18T08:51:15.488 INFO:teuthology.orchestra.run.trial118.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-18T08:51:15.577 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-18T08:51:15.856 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-18T08:51:16.249 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-test (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:16.250 INFO:teuthology.orchestra.run.trial118.stdout:Setting up radosgw (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:16.502 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-18T08:51:16.502 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-18T08:51:16.912 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-base (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:17.030 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-18T08:51:17.425 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-mds (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:17.485 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-18T08:51:17.485 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-18T08:51:17.871 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-mgr (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:17.927 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-18T08:51:17.928 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-18T08:51:18.328 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-mgr-k8sevents (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:18.328 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-osd-classic (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:18.345 INFO:teuthology.orchestra.run.trial118.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-18T08:51:18.354 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:18.365 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-mon (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:18.436 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-18T08:51:18.437 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-18T08:51:18.843 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-mgr-cephadm (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:18.853 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-mgr-dashboard (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:18.864 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-osd (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:18.952 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-18T08:51:18.952 INFO:teuthology.orchestra.run.trial118.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-18T08:51:19.356 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:19.357 INFO:teuthology.orchestra.run.trial118.stdout:Setting up ceph-volume (20.3.0-5271-g32585015-1jammy) ... 2026-02-18T08:51:19.465 INFO:teuthology.orchestra.run.trial118.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-18T08:51:19.549 INFO:teuthology.orchestra.run.trial118.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.11) ... 2026-02-18T08:51:20.217 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:51:20.218 INFO:teuthology.orchestra.run.trial118.stdout:Running kernel seems to be up-to-date. 2026-02-18T08:51:20.218 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:51:20.218 INFO:teuthology.orchestra.run.trial118.stdout:The processor microcode seems to be up-to-date. 2026-02-18T08:51:20.218 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:51:20.218 INFO:teuthology.orchestra.run.trial118.stdout:No services need to be restarted. 2026-02-18T08:51:20.218 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:51:20.218 INFO:teuthology.orchestra.run.trial118.stdout:No containers need to be restarted. 2026-02-18T08:51:20.218 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:51:20.219 INFO:teuthology.orchestra.run.trial118.stdout:No user sessions are running outdated binaries. 2026-02-18T08:51:20.219 INFO:teuthology.orchestra.run.trial118.stdout: 2026-02-18T08:51:20.219 INFO:teuthology.orchestra.run.trial118.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-18T08:51:20.856 INFO:teuthology.orchestra.run.trial118.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-18T08:51:20.862 DEBUG:teuthology.parallel:result is None 2026-02-18T08:51:20.863 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=325850157d309f65985eea9ec9bb9f1d56726fee 2026-02-18T08:51:21.026 DEBUG:teuthology.orchestra.run.trial118:> dpkg-query -W -f '${Version}' ceph 2026-02-18T08:51:21.037 INFO:teuthology.orchestra.run.trial118.stdout:20.3.0-5271-g32585015-1jammy 2026-02-18T08:51:21.037 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5271-g32585015-1jammy 2026-02-18T08:51:21.037 INFO:teuthology.task.install:The correct ceph version 20.3.0-5271-g32585015-1jammy is installed. 2026-02-18T08:51:21.039 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-18T08:51:21.039 DEBUG:teuthology.orchestra.run.trial118:> set -ex 2026-02-18T08:51:21.039 DEBUG:teuthology.orchestra.run.trial118:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-18T08:51:21.087 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-18T08:51:21.087 DEBUG:teuthology.orchestra.run.trial118:> set -ex 2026-02-18T08:51:21.087 DEBUG:teuthology.orchestra.run.trial118:> sudo dd of=/usr/bin/daemon-helper 2026-02-18T08:51:21.139 DEBUG:teuthology.orchestra.run.trial118:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-18T08:51:21.189 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-18T08:51:21.190 DEBUG:teuthology.orchestra.run.trial118:> set -ex 2026-02-18T08:51:21.190 DEBUG:teuthology.orchestra.run.trial118:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-18T08:51:21.243 DEBUG:teuthology.orchestra.run.trial118:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-18T08:51:21.297 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-18T08:51:21.298 DEBUG:teuthology.orchestra.run.trial118:> set -ex 2026-02-18T08:51:21.298 DEBUG:teuthology.orchestra.run.trial118:> sudo dd of=/usr/bin/stdin-killer 2026-02-18T08:51:21.347 DEBUG:teuthology.orchestra.run.trial118:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-18T08:51:21.397 INFO:teuthology.run_tasks:Running task workunit... 2026-02-18T08:51:21.425 INFO:tasks.workunit:Pulling workunits from ref 325850157d309f65985eea9ec9bb9f1d56726fee 2026-02-18T08:51:21.426 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-18T08:51:21.426 INFO:tasks.workunit:timeout=3h 2026-02-18T08:51:21.426 INFO:tasks.workunit:cleanup=True 2026-02-18T08:51:21.426 DEBUG:teuthology.orchestra.run.trial118:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-18T08:51:21.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-18T08:51:21.442 INFO:teuthology.orchestra.run.trial118.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-18T08:51:21.442 DEBUG:teuthology.orchestra.run.trial118:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-18T08:51:21.489 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-18T08:51:21.490 DEBUG:teuthology.orchestra.run.trial118:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-18T08:51:21.534 DEBUG:teuthology.orchestra.run.trial118:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 325850157d309f65985eea9ec9bb9f1d56726fee 2026-02-18T08:51:21.584 INFO:tasks.workunit.client.0.trial118.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-18T08:51:43.360 INFO:tasks.workunit.client.0.trial118.stderr:Note: switching to '325850157d309f65985eea9ec9bb9f1d56726fee'. 2026-02-18T08:51:43.360 INFO:tasks.workunit.client.0.trial118.stderr: 2026-02-18T08:51:43.360 INFO:tasks.workunit.client.0.trial118.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-18T08:51:43.360 INFO:tasks.workunit.client.0.trial118.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-18T08:51:43.360 INFO:tasks.workunit.client.0.trial118.stderr:state without impacting any branches by switching back to a branch. 2026-02-18T08:51:43.360 INFO:tasks.workunit.client.0.trial118.stderr: 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr: 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr: git switch -c 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr: 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr:Or undo this operation with: 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr: 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr: git switch - 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr: 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr: 2026-02-18T08:51:43.361 INFO:tasks.workunit.client.0.trial118.stderr:HEAD is now at 325850157d3 Merge branch 'issue73891' of https://github.com/bill-scales/ceph into wip-bharath18-testing-2026-02-13-0856 2026-02-18T08:51:43.370 DEBUG:teuthology.orchestra.run.trial118:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-18T08:51:43.415 DEBUG:teuthology.orchestra.run.trial118:> set -ex 2026-02-18T08:51:43.416 DEBUG:teuthology.orchestra.run.trial118:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-18T08:51:43.462 INFO:tasks.workunit:Running workunits matching misc on client.0... 2026-02-18T08:51:43.463 INFO:tasks.workunit:Running workunit misc/mclock-config.sh... 2026-02-18T08:51:43.463 DEBUG:teuthology.orchestra.run.trial118:workunit test misc/mclock-config.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh 2026-02-18T08:51:43.516 INFO:tasks.workunit.client.0.trial118.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-18T08:51:43.521 INFO:tasks.workunit.client.0.trial118.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-18T08:51:43.521 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-18T08:51:43.521 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-18T08:51:43.521 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-18T08:51:43.521 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-18T08:51:43.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-18T08:51:43.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-18T08:51:43.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-18T08:51:43.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-18T08:51:43.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/mclock-config 2026-02-18T08:51:43.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:21: run: local dir=td/mclock-config 2026-02-18T08:51:43.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:22: run: shift 2026-02-18T08:51:43.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: export CEPH_MON=127.0.0.1:7124 2026-02-18T08:51:43.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: CEPH_MON=127.0.0.1:7124 2026-02-18T08:51:43.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:25: run: export CEPH_ARGS 2026-02-18T08:51:43.522 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: uuidgen 2026-02-18T08:51:43.523 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: CEPH_ARGS+='--fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none ' 2026-02-18T08:51:43.523 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2026-02-18T08:51:43.523 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:28: run: CEPH_ARGS+='--debug-mclock 20 ' 2026-02-18T08:51:43.524 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: set 2026-02-18T08:51:43.524 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-18T08:51:43.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: local 'funcs=TEST_backfill_limit_adjustment_mclock 2026-02-18T08:51:43.525 INFO:tasks.workunit.client.0.trial118.stderr:TEST_profile_builtin_to_custom 2026-02-18T08:51:43.525 INFO:tasks.workunit.client.0.trial118.stderr:TEST_profile_custom_to_builtin 2026-02-18T08:51:43.525 INFO:tasks.workunit.client.0.trial118.stderr:TEST_recovery_limit_adjustment_mclock' 2026-02-18T08:51:43.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-18T08:51:43.526 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-18T08:51:43.526 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-18T08:51:43.526 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-18T08:51:43.526 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:51:43.526 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:51:43.526 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:51:43.526 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:51:43.526 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:51:43.526 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:51:43.526 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:51:43.526 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:51:43.527 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:51:43.527 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:51:43.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:51:43.529 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:51:43.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:51:43.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:51:43.530 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:51:43.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:51:43.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:51:43.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:51:43.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:51:43.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:51:43.534 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:51:43.535 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:51:43.536 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:51:43.536 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:43.536 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:43.536 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:51:43.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:51:43.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:51:43.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-18T08:51:43.539 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:51:43.539 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:43.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:43.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7523 2026-02-18T08:51:43.541 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:51:43.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-18T08:51:43.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-18T08:51:43.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:51:43.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-18T08:51:43.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_backfill_limit_adjustment_mclock td/mclock-config 2026-02-18T08:51:43.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:236: TEST_backfill_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-18T08:51:43.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:238: TEST_backfill_limit_adjustment_mclock: setup td/mclock-config 2026-02-18T08:51:43.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-18T08:51:43.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-18T08:51:43.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:51:43.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:51:43.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:51:43.544 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:51:43.544 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:51:43.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:51:43.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:51:43.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:51:43.545 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:51:43.545 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:51:43.546 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:51:43.546 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:51:43.547 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:51:43.547 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:51:43.548 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:51:43.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:51:43.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:51:43.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:51:43.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:51:43.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:51:43.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:51:43.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:51:43.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:51:43.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:43.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:43.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:51:43.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:51:43.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:51:43.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-18T08:51:43.556 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:51:43.556 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:43.557 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:43.557 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7523 2026-02-18T08:51:43.558 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:51:43.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:51:43.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:51:43.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-18T08:51:43.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:239: TEST_backfill_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-18T08:51:43.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-18T08:51:43.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:51:43.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:51:43.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:51:43.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-18T08:51:43.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-18T08:51:43.594 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:51:43.594 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:51:43.594 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:51:43.594 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:51:43.594 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:43.594 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:43.595 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:51:43.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:51:43.623 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:51:43.624 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:51:43.624 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:51:43.624 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:51:43.624 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:51:43.625 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:51:43.626 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:51:43.626 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:51:43.626 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:51:43.626 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:43.626 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:43.626 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:43.626 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-mon.a.asok 2026-02-18T08:51:43.626 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:51:43.626 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7523/ceph-mon.a.asok config get fsid 2026-02-18T08:51:43.681 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:51:43.681 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:51:43.681 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:51:43.681 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:51:43.682 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:51:43.682 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:51:43.682 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:51:43.682 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:51:43.683 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:43.683 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:43.683 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:43.683 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-mon.a.asok 2026-02-18T08:51:43.683 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:51:43.683 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7523/ceph-mon.a.asok config get mon_host 2026-02-18T08:51:43.736 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:240: TEST_backfill_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-18T08:51:43.736 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-18T08:51:43.736 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:51:43.736 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:51:43.736 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:51:43.736 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-18T08:51:43.736 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:51:43.846 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:51:43.847 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:51:43.847 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:51:43.847 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:51:43.847 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:43.847 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:43.847 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:51:43.848 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:51:43.849 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:51:43.870 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:242: TEST_backfill_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-18T08:51:43.870 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-18T08:51:43.870 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:51:43.870 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:51:43.870 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:51:43.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-18T08:51:43.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-18T08:51:43.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:51:43.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:51:43.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:51:43.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-18T08:51:43.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-18T08:51:43.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:51:43.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:51:43.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-18T08:51:43.871 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:51:43.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:51:43.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:51:43.872 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:51:43.872 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:43.872 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:43.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:51:43.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:51:43.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:51:43.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:51:43.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:51:43.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-18T08:51:43.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-18T08:51:43.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:51:43.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:51:43.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:51:43.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:51:43.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:51:43.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-18T08:51:43.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-18T08:51:43.874 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:51:43.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4109e119-2d47-405c-942a-2882cfd8a321 2026-02-18T08:51:43.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 4109e119-2d47-405c-942a-2882cfd8a321' 2026-02-18T08:51:43.874 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 4109e119-2d47-405c-942a-2882cfd8a321 2026-02-18T08:51:43.875 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:51:43.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCffZVpeBtnNBAAMn+yON6hIzLHLBj8g4v5Eg== 2026-02-18T08:51:43.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCffZVpeBtnNBAAMn+yON6hIzLHLBj8g4v5Eg=="}' 2026-02-18T08:51:43.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4109e119-2d47-405c-942a-2882cfd8a321 -i td/mclock-config/0/new.json 2026-02-18T08:51:43.992 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:51:44.001 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-18T08:51:44.002 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQCffZVpeBtnNBAAMn+yON6hIzLHLBj8g4v5Eg== --osd-uuid 4109e119-2d47-405c-942a-2882cfd8a321 2026-02-18T08:51:44.020 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:44.013+0000 7fad34f1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:44.022 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:44.017+0000 7fad34f1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:44.023 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:44.017+0000 7fad34f1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:44.024 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:44.017+0000 7fad34f1fa40 -1 bdev(0x557eb2f96400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:51:44.024 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:44.017+0000 7fad34f1fa40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-18T08:51:44.759 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-18T08:51:44.759 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:51:44.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:51:44.760 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:51:44.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:51:44.878 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:51:44.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:51:44.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-18T08:51:44.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:51:44.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:51:44.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:51:44.903 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:44.897+0000 7f6f12100a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:44.906 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:44.897+0000 7f6f12100a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:44.907 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:44.901+0000 7f6f12100a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:44.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:51:44.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:51:44.988 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:51:44.988 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:51:44.988 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:51:44.988 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:51:44.988 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:51:44.988 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:51:44.989 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:51:44.989 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:51:45.096 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:51:45.432 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:45.425+0000 7f6f12100a40 -1 Falling back to public interface 2026-02-18T08:51:46.097 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:51:46.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:51:46.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:51:46.098 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:51:46.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:51:46.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:51:46.202 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:51:46.415 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:46.409+0000 7f6f12100a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:51:47.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:51:47.204 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:51:47.204 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:51:47.205 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:51:47.205 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:51:47.205 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:51:47.413 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1851698915,v1:127.0.0.1:6801/1851698915] [v2:127.0.0.1:6802/1851698915,v1:127.0.0.1:6803/1851698915] exists,up 4109e119-2d47-405c-942a-2882cfd8a321 2026-02-18T08:51:47.413 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:51:47.413 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:51:47.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:51:47.414 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-18T08:51:47.414 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: bc 2026-02-18T08:51:47.416 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:51:47.416 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:47.416 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:47.416 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:47.416 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:47.416 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:47.416 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:47.416 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:51:47.416 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_max_backfills 2026-02-18T08:51:47.478 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: local backfills=1 2026-02-18T08:51:47.478 INFO:tasks.workunit.client.0.trial118.stdout:osd_max_backfills: 1 2026-02-18T08:51:47.478 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:246: TEST_backfill_limit_adjustment_mclock: echo 'osd_max_backfills: 1' 2026-02-18T08:51:47.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:251: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-18T08:51:47.690 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:252: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-18T08:51:49.693 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-18T08:51:49.693 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: bc 2026-02-18T08:51:49.694 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:51:49.694 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:49.694 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:49.695 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:49.695 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:49.695 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:49.695 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:49.695 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:51:49.695 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_max_backfills 2026-02-18T08:51:49.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: local max_backfills=1 2026-02-18T08:51:49.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:255: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-18T08:51:49.757 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-18T08:51:49.757 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: bc 2026-02-18T08:51:49.758 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:51:49.759 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:49.759 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:49.759 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:49.759 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:49.759 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:49.759 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:49.760 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:51:49.760 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok dump_recovery_reservations 2026-02-18T08:51:49.818 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-18T08:51:49.818 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:260: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-18T08:51:49.819 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-18T08:51:49.819 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: bc 2026-02-18T08:51:49.820 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:51:49.820 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:49.820 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:49.820 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:49.820 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:49.820 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:49.820 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:49.820 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:51:49.820 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok dump_recovery_reservations 2026-02-18T08:51:49.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-18T08:51:49.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:263: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-18T08:51:49.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:267: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-18T08:51:50.093 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:268: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-18T08:51:50.320 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:269: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-18T08:51:52.323 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-18T08:51:52.323 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: bc 2026-02-18T08:51:52.324 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:51:52.324 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:52.324 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:52.324 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:52.324 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:52.325 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:52.325 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:52.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:51:52.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_max_backfills 2026-02-18T08:51:52.387 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-18T08:51:52.387 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:272: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-18T08:51:52.388 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-18T08:51:52.388 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: bc 2026-02-18T08:51:52.389 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:51:52.389 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:52.389 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:52.389 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:52.389 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:52.389 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:52.389 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:52.390 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:51:52.390 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok dump_recovery_reservations 2026-02-18T08:51:52.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-18T08:51:52.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:277: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-18T08:51:52.450 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-18T08:51:52.450 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: bc 2026-02-18T08:51:52.451 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:51:52.451 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:52.451 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:52.452 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:52.452 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:52.452 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:52.452 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:52.452 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:51:52.452 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok dump_recovery_reservations 2026-02-18T08:51:52.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-18T08:51:52.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:280: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-18T08:51:52.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:284: TEST_backfill_limit_adjustment_mclock: kill_daemons td/mclock-config TERM osd 2026-02-18T08:51:52.511 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:51:52.511 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:51:52.512 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:51:52.512 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:51:52.512 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:51:52.820 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:51:52.820 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:285: TEST_backfill_limit_adjustment_mclock: ceph osd down 0 2026-02-18T08:51:53.035 INFO:tasks.workunit.client.0.trial118.stderr:osd.0 is already down. 2026-02-18T08:51:53.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:286: TEST_backfill_limit_adjustment_mclock: wait_for_osd down 0 2026-02-18T08:51:53.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-18T08:51:53.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:51:53.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:51:53.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:51:53.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:51:53.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:51:53.048 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:51:53.048 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:51:53.048 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-18T08:51:53.261 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/1851698915,v1:127.0.0.1:6801/1851698915] [v2:127.0.0.1:6802/1851698915,v1:127.0.0.1:6803/1851698915] exists 4109e119-2d47-405c-942a-2882cfd8a321 2026-02-18T08:51:53.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:51:53.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:51:53.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:51:53.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:287: TEST_backfill_limit_adjustment_mclock: activate_osd td/mclock-config 0 --osd-op-queue=mclock_scheduler 2026-02-18T08:51:53.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/mclock-config 2026-02-18T08:51:53.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/mclock-config/0 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-18T08:51:53.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:51:53.266 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:51:53.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:51:53.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:51:53.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-18T08:51:53.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=--osd-op-queue=mclock_scheduler 2026-02-18T08:51:53.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/mclock-config/0 2026-02-18T08:51:53.267 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:51:53.268 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-18T08:51:53.268 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --debug-osd=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler 2026-02-18T08:51:53.268 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/mclock-config/0/whoami 2026-02-18T08:51:53.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-18T08:51:53.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-18T08:51:53.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-18T08:51:53.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-18T08:51:53.289 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:53.281+0000 7f5adb60da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:53.291 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:53.285+0000 7f5adb60da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:53.292 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:53.285+0000 7f5adb60da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:53.481 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-18T08:51:53.481 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:51:53.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:51:53.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:51:53.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:51:53.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:51:53.482 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:51:53.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:51:53.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:51:53.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:51:53.664 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:53.657+0000 7f5adb60da40 -1 Falling back to public interface 2026-02-18T08:51:53.688 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:51:54.288 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:54.281+0000 7f5adb60da40 -1 osd.0 6 log_to_monitors true 2026-02-18T08:51:54.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:51:54.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:51:54.690 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:51:54.690 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:51:54.690 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:51:54.691 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:51:54.896 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 9 up_thru 0 down_at 6 last_clean_interval [5,6) [v2:127.0.0.1:6800/1270366475,v1:127.0.0.1:6801/1270366475] [v2:127.0.0.1:6802/1270366475,v1:127.0.0.1:6803/1270366475] exists,up 4109e119-2d47-405c-942a-2882cfd8a321 2026-02-18T08:51:54.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:51:54.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:51:54.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:51:54.898 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-18T08:51:54.898 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: bc 2026-02-18T08:51:54.898 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:51:54.898 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:54.898 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:54.898 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:54.898 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:54.899 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:54.899 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:54.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:51:54.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_max_backfills 2026-02-18T08:51:54.960 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-18T08:51:54.960 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:291: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-18T08:51:54.961 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-18T08:51:54.961 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: bc 2026-02-18T08:51:54.961 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:51:54.962 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:54.962 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:54.962 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:54.962 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:54.962 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:54.962 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:54.963 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:51:54.963 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok dump_recovery_reservations 2026-02-18T08:51:55.022 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-18T08:51:55.022 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:296: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-18T08:51:55.023 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-18T08:51:55.023 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: bc 2026-02-18T08:51:55.023 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:51:55.024 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:55.024 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:55.024 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:55.024 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:55.024 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:55.024 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:55.025 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:51:55.025 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok dump_recovery_reservations 2026-02-18T08:51:55.084 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-18T08:51:55.084 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:299: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-18T08:51:55.084 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:301: TEST_backfill_limit_adjustment_mclock: teardown td/mclock-config 2026-02-18T08:51:55.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:51:55.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:51:55.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:51:55.085 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:51:55.085 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:51:55.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:51:55.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:51:55.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:51:55.192 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:51:55.192 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:51:55.192 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:51:55.193 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:51:55.193 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:51:55.194 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:51:55.194 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:51:55.195 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:51:55.195 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:51:55.196 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:51:55.196 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:51:55.197 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:51:55.198 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:51:55.198 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:51:55.203 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:51:55.203 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:55.203 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:55.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:51:55.205 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:51:55.205 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:51:55.205 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-18T08:51:55.205 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:51:55.206 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:51:55.206 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:51:55.206 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:51:55.206 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:51:55.206 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:51:55.207 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:51:55.207 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:51:55.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:51:55.208 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:51:55.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:51:55.210 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:51:55.211 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:51:55.211 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:51:55.211 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:51:55.212 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:51:55.212 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:51:55.213 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:51:55.213 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:51:55.214 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:51:55.214 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:51:55.215 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:51:55.216 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:51:55.216 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:55.216 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:55.216 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:51:55.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:51:55.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:51:55.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-18T08:51:55.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-18T08:51:55.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-18T08:51:55.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-18T08:51:55.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:51:55.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:51:55.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:51:55.219 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:51:55.219 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:51:55.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:51:55.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:51:55.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:51:55.220 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:51:55.220 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:51:55.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:51:55.221 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:51:55.222 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:51:55.222 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:51:55.223 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:51:55.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:51:55.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:51:55.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:51:55.225 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:51:55.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:51:55.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:51:55.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:51:55.228 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:51:55.228 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:55.228 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:55.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:51:55.230 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:51:55.230 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:51:55.230 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-18T08:51:55.231 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:51:55.231 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:55.231 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:55.231 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7523 2026-02-18T08:51:55.233 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:51:55.234 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:51:55.234 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:51:55.234 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-18T08:51:55.234 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_builtin_to_custom td/mclock-config 2026-02-18T08:51:55.234 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:39: TEST_profile_builtin_to_custom: local dir=td/mclock-config 2026-02-18T08:51:55.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:41: TEST_profile_builtin_to_custom: run_mon td/mclock-config a 2026-02-18T08:51:55.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-18T08:51:55.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:51:55.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:51:55.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:51:55.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-18T08:51:55.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-18T08:51:55.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:51:55.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:51:55.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:51:55.255 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:51:55.255 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:55.255 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:55.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:51:55.256 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:51:55.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:51:55.283 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:51:55.283 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:51:55.283 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:51:55.283 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:51:55.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:51:55.284 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:51:55.284 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:51:55.284 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:51:55.284 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:55.284 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:55.285 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:55.285 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-mon.a.asok 2026-02-18T08:51:55.285 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:51:55.285 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7523/ceph-mon.a.asok config get fsid 2026-02-18T08:51:55.340 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:51:55.341 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:51:55.341 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:51:55.341 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:51:55.341 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:51:55.342 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:51:55.342 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:51:55.342 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:51:55.342 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:55.342 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:55.343 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:55.343 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-mon.a.asok 2026-02-18T08:51:55.343 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:51:55.343 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7523/ceph-mon.a.asok config get mon_host 2026-02-18T08:51:55.397 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:42: TEST_profile_builtin_to_custom: run_mgr td/mclock-config x 2026-02-18T08:51:55.397 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-18T08:51:55.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:51:55.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:51:55.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:51:55.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-18T08:51:55.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:51:55.508 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:51:55.508 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:51:55.508 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:51:55.508 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:51:55.508 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:55.509 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:55.509 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:51:55.509 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:51:55.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:51:55.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:43: TEST_profile_builtin_to_custom: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-18T08:51:55.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-18T08:51:55.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:51:55.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:51:55.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:51:55.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-18T08:51:55.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-18T08:51:55.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:51:55.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:51:55.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:51:55.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:51:55.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:51:55.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:51:55.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:51:55.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-18T08:51:55.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-18T08:51:55.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:51:55.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:51:55.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:51:55.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:51:55.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:51:55.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-18T08:51:55.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-18T08:51:55.533 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:51:55.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=67d07c9b-5cdb-4f99-91a7-c76bf37afddc 2026-02-18T08:51:55.533 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 67d07c9b-5cdb-4f99-91a7-c76bf37afddc 2026-02-18T08:51:55.534 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 67d07c9b-5cdb-4f99-91a7-c76bf37afddc' 2026-02-18T08:51:55.534 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:51:55.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCrfZVpUqUXIBAA/Ff5nCzZZTGs7ADtZqrY3g== 2026-02-18T08:51:55.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCrfZVpUqUXIBAA/Ff5nCzZZTGs7ADtZqrY3g=="}' 2026-02-18T08:51:55.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 67d07c9b-5cdb-4f99-91a7-c76bf37afddc -i td/mclock-config/0/new.json 2026-02-18T08:51:55.652 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:51:55.659 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-18T08:51:55.660 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQCrfZVpUqUXIBAA/Ff5nCzZZTGs7ADtZqrY3g== --osd-uuid 67d07c9b-5cdb-4f99-91a7-c76bf37afddc 2026-02-18T08:51:55.676 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:55.669+0000 7f525a511a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:55.678 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:55.669+0000 7f525a511a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:55.678 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:55.673+0000 7f525a511a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:55.679 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:55.673+0000 7f525a511a40 -1 bdev(0x55915b1a8400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:51:55.679 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:55.673+0000 7f525a511a40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-18T08:51:56.346 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-18T08:51:56.346 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:51:56.347 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:51:56.347 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:51:56.347 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:51:56.458 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:51:56.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:51:56.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-18T08:51:56.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:51:56.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:51:56.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:51:56.472 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:56.465+0000 7f5b57379a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:56.474 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:56.465+0000 7f5b57379a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:56.475 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:56.469+0000 7f5b57379a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:51:56.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:51:56.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:51:56.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:51:56.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:51:56.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:51:56.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:51:56.573 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:51:56.573 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:51:56.573 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:51:56.573 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:51:56.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:51:56.856 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:56.849+0000 7f5b57379a40 -1 Falling back to public interface 2026-02-18T08:51:57.519 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:51:57.513+0000 7f5b57379a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:51:57.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:51:57.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:51:57.681 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:51:57.681 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:51:57.682 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:51:57.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:51:57.834 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:51:58.835 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:51:58.835 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:51:58.835 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:51:58.836 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:51:58.836 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:51:58.836 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:51:59.054 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1238405508,v1:127.0.0.1:6801/1238405508] [v2:127.0.0.1:6802/1238405508,v1:127.0.0.1:6803/1238405508] exists,up 67d07c9b-5cdb-4f99-91a7-c76bf37afddc 2026-02-18T08:51:59.055 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:51:59.055 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:51:59.055 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:51:59.055 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_profile 2026-02-18T08:51:59.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: local mclock_profile=balanced 2026-02-18T08:51:59.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:47: TEST_profile_builtin_to_custom: test balanced = balanced 2026-02-18T08:51:59.269 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-18T08:51:59.270 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-18T08:51:59.270 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:59.270 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:59.271 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:59.271 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:59.271 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:59.271 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:59.271 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-18T08:51:59.271 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_mclock_profile 2026-02-18T08:51:59.331 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: mclock_profile='"high_recovery_ops"' 2026-02-18T08:51:59.332 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: eval echo '"high_recovery_ops"' 2026-02-18T08:51:59.332 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: echo high_recovery_ops 2026-02-18T08:51:59.332 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: mclock_profile=high_recovery_ops 2026-02-18T08:51:59.332 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:54: TEST_profile_builtin_to_custom: test high_recovery_ops = high_recovery_ops 2026-02-18T08:51:59.332 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:57: TEST_profile_builtin_to_custom: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-18T08:51:59.439 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:51:59.440 INFO:tasks.workunit.client.0.trial118.stdout: "success": "" 2026-02-18T08:51:59.440 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:51:59.447 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-18T08:51:59.448 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-18T08:51:59.448 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:59.448 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:59.448 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:59.448 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:59.449 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:59.449 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:59.449 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-18T08:51:59.449 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_mclock_profile 2026-02-18T08:51:59.509 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: mclock_profile='"custom"' 2026-02-18T08:51:59.510 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: eval echo '"custom"' 2026-02-18T08:51:59.510 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: echo custom 2026-02-18T08:51:59.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: mclock_profile=custom 2026-02-18T08:51:59.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: test custom = custom 2026-02-18T08:51:59.511 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-18T08:51:59.511 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: bc 2026-02-18T08:51:59.512 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-18T08:51:59.512 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:51:59.512 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:51:59.512 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:51:59.512 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:51:59.512 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:51:59.512 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:51:59.513 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-18T08:51:59.513 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-18T08:51:59.573 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: local client_res=0.000000 2026-02-18T08:51:59.574 INFO:tasks.workunit.client.0.trial118.stdout:client_res = 0.000000 2026-02-18T08:51:59.575 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:69: TEST_profile_builtin_to_custom: echo 'client_res = 0.000000' 2026-02-18T08:51:59.575 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: echo '0.000000 + 0.1' 2026-02-18T08:51:59.575 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: bc -l 2026-02-18T08:51:59.575 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: local client_res_new=.100000 2026-02-18T08:51:59.576 INFO:tasks.workunit.client.0.trial118.stdout:client_res_new = .100000 2026-02-18T08:51:59.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:71: TEST_profile_builtin_to_custom: echo 'client_res_new = .100000' 2026-02-18T08:51:59.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:72: TEST_profile_builtin_to_custom: ceph config set osd.0 osd_mclock_scheduler_client_res .100000 2026-02-18T08:51:59.800 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-18T08:52:00.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: local res=0.100000 2026-02-18T08:52:00.030 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: echo '0.100000 != .100000' 2026-02-18T08:52:00.030 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: bc -l 2026-02-18T08:52:00.031 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-18T08:52:00.032 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-18T08:52:00.032 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: bc 2026-02-18T08:52:00.033 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-18T08:52:00.033 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:52:00.033 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:52:00.033 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:00.033 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.034 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.034 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:52:00.034 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-18T08:52:00.034 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-18T08:52:00.096 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: res=0.100000 2026-02-18T08:52:00.096 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: echo '0.100000 != .100000' 2026-02-18T08:52:00.097 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: bc -l 2026-02-18T08:52:00.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-18T08:52:00.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:89: TEST_profile_builtin_to_custom: teardown td/mclock-config 2026-02-18T08:52:00.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:52:00.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:52:00.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:52:00.099 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:00.099 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:00.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:00.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:00.100 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:00.206 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:00.207 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:00.207 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:00.208 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:00.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:00.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:00.209 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:00.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:00.211 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:00.211 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:00.211 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:00.212 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:00.213 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:52:00.213 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:52:00.216 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:00.216 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.216 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.216 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:52:00.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:00.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:00.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-18T08:52:00.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:52:00.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:52:00.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:52:00.219 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:00.219 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:00.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:00.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:00.219 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:00.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:00.221 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:00.222 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:00.222 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:00.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:00.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:00.223 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:00.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:00.225 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:00.225 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:00.225 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:00.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:00.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:52:00.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:52:00.229 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:00.229 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.229 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:52:00.231 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:00.231 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:00.231 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-18T08:52:00.231 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-18T08:52:00.231 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-18T08:52:00.231 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-18T08:52:00.231 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:52:00.231 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:52:00.232 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:52:00.232 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:00.232 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:00.232 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:00.232 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:00.232 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:00.233 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:00.233 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:00.234 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:00.234 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:00.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:00.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:00.236 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:00.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:00.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:00.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:00.237 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:00.238 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:00.239 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:52:00.239 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:52:00.241 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:00.241 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.241 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.241 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:52:00.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:00.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:00.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-18T08:52:00.244 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:52:00.244 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.244 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.244 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7523 2026-02-18T08:52:00.246 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:52:00.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_custom_to_builtin td/mclock-config 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:93: TEST_profile_custom_to_builtin: local dir=td/mclock-config 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:95: TEST_profile_custom_to_builtin: setup td/mclock-config 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:00.248 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:00.249 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:00.249 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:00.249 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:00.249 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:00.250 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:00.251 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:00.251 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:00.252 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:00.252 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:00.253 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:00.254 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:00.254 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:00.254 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:00.254 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:00.255 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:00.256 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:52:00.256 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:52:00.257 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:00.257 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.257 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.257 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:52:00.258 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:00.259 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:00.259 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-18T08:52:00.260 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:52:00.260 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.260 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.260 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7523 2026-02-18T08:52:00.262 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:52:00.263 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:52:00.263 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:52:00.263 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-18T08:52:00.263 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:96: TEST_profile_custom_to_builtin: run_mon td/mclock-config a 2026-02-18T08:52:00.263 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-18T08:52:00.263 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:52:00.263 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:52:00.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:52:00.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-18T08:52:00.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-18T08:52:00.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:52:00.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:52:00.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:52:00.285 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:52:00.285 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.285 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.286 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:52:00.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:52:00.313 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:52:00.314 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:52:00.314 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:52:00.314 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:52:00.314 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:52:00.315 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:52:00.315 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:52:00.315 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:52:00.316 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:52:00.316 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:00.316 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.316 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.316 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-mon.a.asok 2026-02-18T08:52:00.316 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:52:00.316 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7523/ceph-mon.a.asok config get fsid 2026-02-18T08:52:00.371 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:52:00.371 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:52:00.371 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:52:00.371 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:52:00.372 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:52:00.373 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:52:00.373 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:52:00.373 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:52:00.373 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:00.373 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.373 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.373 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-mon.a.asok 2026-02-18T08:52:00.373 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:52:00.374 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7523/ceph-mon.a.asok config get mon_host 2026-02-18T08:52:00.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:97: TEST_profile_custom_to_builtin: run_mgr td/mclock-config x 2026-02-18T08:52:00.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-18T08:52:00.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:52:00.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:52:00.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:52:00.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-18T08:52:00.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:52:00.538 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:52:00.539 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:52:00.539 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:52:00.539 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:52:00.539 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.540 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:52:00.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:52:00.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:52:00.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:98: TEST_profile_custom_to_builtin: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-18T08:52:00.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-18T08:52:00.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:52:00.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:52:00.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:52:00.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-18T08:52:00.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-18T08:52:00.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:52:00.562 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:52:00.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:52:00.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-18T08:52:00.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-18T08:52:00.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:52:00.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=49f153f2-48e3-4926-8a64-68285f0c3489 2026-02-18T08:52:00.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 49f153f2-48e3-4926-8a64-68285f0c3489' 2026-02-18T08:52:00.565 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 49f153f2-48e3-4926-8a64-68285f0c3489 2026-02-18T08:52:00.565 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:52:00.574 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCwfZVpYyP7IRAAWOT0qlIiu1DLPVtBVf3lsQ== 2026-02-18T08:52:00.574 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCwfZVpYyP7IRAAWOT0qlIiu1DLPVtBVf3lsQ=="}' 2026-02-18T08:52:00.574 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 49f153f2-48e3-4926-8a64-68285f0c3489 -i td/mclock-config/0/new.json 2026-02-18T08:52:00.682 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:52:00.690 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-18T08:52:00.691 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQCwfZVpYyP7IRAAWOT0qlIiu1DLPVtBVf3lsQ== --osd-uuid 49f153f2-48e3-4926-8a64-68285f0c3489 2026-02-18T08:52:00.707 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:00.700+0000 7f867edc5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:00.708 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:00.700+0000 7f867edc5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:00.709 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:00.700+0000 7f867edc5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:00.710 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:00.700+0000 7f867edc5a40 -1 bdev(0x560672320400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:52:00.710 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:00.700+0000 7f867edc5a40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-18T08:52:01.388 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-18T08:52:01.389 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:52:01.390 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:52:01.390 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:52:01.390 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:52:01.503 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:52:01.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:52:01.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-18T08:52:01.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:52:01.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:52:01.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:52:01.518 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:01.512+0000 7f7efe728a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:01.520 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:01.512+0000 7f7efe728a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:01.521 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:01.512+0000 7f7efe728a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:01.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:52:01.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:52:01.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:52:01.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:52:01.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:52:01.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:01.616 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:52:01.616 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:52:01.616 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:01.616 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:52:01.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:52:01.964 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:01.956+0000 7f7efe728a40 -1 Falling back to public interface 2026-02-18T08:52:02.680 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:02.672+0000 7f7efe728a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:52:02.722 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:52:02.722 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:52:02.722 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:02.723 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:52:02.723 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:52:02.725 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:02.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:52:03.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:52:03.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:03.863 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:52:03.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:52:03.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:03.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:52:04.074 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1820456229,v1:127.0.0.1:6801/1820456229] [v2:127.0.0.1:6802/1820456229,v1:127.0.0.1:6803/1820456229] exists,up 49f153f2-48e3-4926-8a64-68285f0c3489 2026-02-18T08:52:04.075 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:52:04.075 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:52:04.075 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:52:04.075 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:101: TEST_profile_custom_to_builtin: local def_mclock_profile 2026-02-18T08:52:04.075 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_profile 2026-02-18T08:52:04.291 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: def_mclock_profile=balanced 2026-02-18T08:52:04.291 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:103: TEST_profile_custom_to_builtin: test balanced = balanced 2026-02-18T08:52:04.292 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-18T08:52:04.293 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-18T08:52:04.293 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:52:04.293 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:52:04.293 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:04.293 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:04.294 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:04.294 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:52:04.294 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-18T08:52:04.294 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_mclock_profile 2026-02-18T08:52:04.351 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: local 'orig_mclock_profile="high_recovery_ops"' 2026-02-18T08:52:04.352 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: eval echo '"high_recovery_ops"' 2026-02-18T08:52:04.352 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: echo high_recovery_ops 2026-02-18T08:52:04.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: orig_mclock_profile=high_recovery_ops 2026-02-18T08:52:04.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:110: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-18T08:52:04.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:113: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-18T08:52:04.449 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:52:04.449 INFO:tasks.workunit.client.0.trial118.stdout: "success": "" 2026-02-18T08:52:04.449 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:52:04.457 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-18T08:52:04.458 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-18T08:52:04.458 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:52:04.458 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:52:04.458 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:04.459 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:04.459 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:04.459 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:52:04.459 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-18T08:52:04.459 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_mclock_profile 2026-02-18T08:52:04.518 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: local 'mclock_profile="custom"' 2026-02-18T08:52:04.519 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: eval echo '"custom"' 2026-02-18T08:52:04.519 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: echo custom 2026-02-18T08:52:04.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: mclock_profile=custom 2026-02-18T08:52:04.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:120: TEST_profile_custom_to_builtin: test custom = custom 2026-02-18T08:52:04.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:123: TEST_profile_custom_to_builtin: local client_res 2026-02-18T08:52:04.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-18T08:52:04.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: bc 2026-02-18T08:52:04.521 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-18T08:52:04.522 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:52:04.522 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:52:04.522 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:04.522 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:04.522 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:04.522 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:52:04.523 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-18T08:52:04.523 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-18T08:52:04.582 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: client_res=0.000000 2026-02-18T08:52:04.582 INFO:tasks.workunit.client.0.trial118.stdout:Original client_res for osd.0 = 0.000000 2026-02-18T08:52:04.583 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: echo 'Original client_res for osd.0 = 0.000000' 2026-02-18T08:52:04.583 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: echo '0.000000 + 0.1' 2026-02-18T08:52:04.583 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: bc -l 2026-02-18T08:52:04.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: local client_res_new=.100000 2026-02-18T08:52:04.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:131: TEST_profile_custom_to_builtin: echo 'client_res_new = .100000' 2026-02-18T08:52:04.584 INFO:tasks.workunit.client.0.trial118.stdout:client_res_new = .100000 2026-02-18T08:52:04.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:132: TEST_profile_custom_to_builtin: ceph config set osd osd_mclock_scheduler_client_res .100000 2026-02-18T08:52:04.803 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-18T08:52:05.032 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: local res=0.100000 2026-02-18T08:52:05.033 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-18T08:52:05.033 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: bc -l 2026-02-18T08:52:05.034 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-18T08:52:05.035 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-18T08:52:05.036 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: bc 2026-02-18T08:52:05.036 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-18T08:52:05.036 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:52:05.036 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:52:05.037 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:05.037 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:05.037 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:05.037 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:52:05.037 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-18T08:52:05.038 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-18T08:52:05.096 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: res=0.100000 2026-02-18T08:52:05.097 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-18T08:52:05.097 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: bc -l 2026-02-18T08:52:05.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-18T08:52:05.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:154: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile high_recovery_ops 2026-02-18T08:52:05.176 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:52:05.176 INFO:tasks.workunit.client.0.trial118.stdout: "success": "" 2026-02-18T08:52:05.177 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:52:05.183 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-18T08:52:05.185 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-18T08:52:05.185 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:52:05.185 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:52:05.185 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:05.185 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:05.185 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:05.186 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:52:05.186 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-18T08:52:05.186 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_mclock_profile 2026-02-18T08:52:05.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: eval 'mclock_profile="high_recovery_ops"' 2026-02-18T08:52:05.247 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: mclock_profile=high_recovery_ops 2026-02-18T08:52:05.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:160: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-18T08:52:05.247 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-18T08:52:05.457 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: local res=0.100000 2026-02-18T08:52:05.458 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-18T08:52:05.458 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: bc -l 2026-02-18T08:52:05.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-18T08:52:05.460 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-18T08:52:05.460 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: bc 2026-02-18T08:52:05.461 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-18T08:52:05.461 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:52:05.461 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:52:05.462 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:05.462 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:05.462 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:05.462 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:52:05.462 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-18T08:52:05.462 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-18T08:52:05.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: res=0.100000 2026-02-18T08:52:05.523 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-18T08:52:05.523 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: bc -l 2026-02-18T08:52:05.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-18T08:52:05.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:178: TEST_profile_custom_to_builtin: ceph config rm osd osd_mclock_scheduler_client_res 2026-02-18T08:52:05.749 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:180: TEST_profile_custom_to_builtin: sleep 5 2026-02-18T08:52:10.751 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-18T08:52:10.969 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: res=0.000000 2026-02-18T08:52:10.969 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: echo '0.000000 != 0.0' 2026-02-18T08:52:10.970 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: bc -l 2026-02-18T08:52:10.971 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-18T08:52:10.971 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-18T08:52:10.972 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: bc 2026-02-18T08:52:10.972 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-18T08:52:10.973 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:52:10.973 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:52:10.973 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:10.973 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:10.973 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:10.973 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:52:10.974 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-18T08:52:10.974 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-18T08:52:11.033 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: res=0.000000 2026-02-18T08:52:11.034 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: echo '0.000000 != 0.000000' 2026-02-18T08:52:11.034 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: bc -l 2026-02-18T08:52:11.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-18T08:52:11.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:198: TEST_profile_custom_to_builtin: teardown td/mclock-config 2026-02-18T08:52:11.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:52:11.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:52:11.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:52:11.037 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:11.037 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:11.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:11.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:11.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:11.144 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:11.144 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:11.145 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:11.146 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:11.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:11.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:11.147 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:11.148 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:11.148 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:11.149 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:11.149 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:11.150 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:11.151 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:52:11.151 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:52:11.155 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:11.155 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:11.155 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:11.155 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:52:11.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:11.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:11.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-18T08:52:11.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:52:11.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:52:11.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:52:11.157 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:11.157 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:11.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:11.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:11.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:11.159 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:11.159 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:11.160 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:11.160 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:11.162 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:11.162 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:11.162 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:11.163 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:11.163 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:11.164 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:11.164 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:11.165 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:11.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:52:11.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:52:11.167 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:11.168 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:11.168 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:11.168 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:52:11.170 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:11.170 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:11.170 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-18T08:52:11.170 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-18T08:52:11.170 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-18T08:52:11.170 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-18T08:52:11.170 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:52:11.170 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:52:11.171 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:52:11.171 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:11.171 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:11.171 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:11.171 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:11.171 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:11.172 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:11.172 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:11.173 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:11.173 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:11.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:11.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:11.175 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:11.176 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:11.176 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:11.177 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:11.177 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:11.178 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:11.179 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:52:11.179 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:52:11.180 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:11.181 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:11.181 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:11.181 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:52:11.182 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:11.182 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:11.182 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-18T08:52:11.183 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:52:11.183 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:11.184 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:11.184 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7523 2026-02-18T08:52:11.185 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:52:11.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:52:11.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:52:11.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-18T08:52:11.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_recovery_limit_adjustment_mclock td/mclock-config 2026-02-18T08:52:11.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:202: TEST_recovery_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-18T08:52:11.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:204: TEST_recovery_limit_adjustment_mclock: setup td/mclock-config 2026-02-18T08:52:11.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-18T08:52:11.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-18T08:52:11.188 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:52:11.188 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:52:11.188 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:52:11.188 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:11.188 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:11.188 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:11.188 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:11.188 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:11.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:11.189 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:11.190 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:11.190 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:11.192 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:11.192 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:11.192 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:11.193 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:11.193 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:11.194 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:11.194 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:11.195 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:11.196 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:52:11.196 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:52:11.197 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:11.197 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:11.197 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:11.197 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:52:11.198 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:11.198 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:11.199 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-18T08:52:11.200 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:52:11.200 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:11.200 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:11.200 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7523 2026-02-18T08:52:11.202 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:52:11.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:52:11.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:52:11.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-18T08:52:11.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:205: TEST_recovery_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-18T08:52:11.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-18T08:52:11.204 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:52:11.204 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:52:11.204 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:52:11.204 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-18T08:52:11.204 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-18T08:52:11.224 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:52:11.224 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:52:11.224 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:52:11.225 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:52:11.225 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:11.225 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:11.225 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:52:11.225 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:52:11.254 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:52:11.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:52:11.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:52:11.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:52:11.256 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:52:11.256 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:52:11.257 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:52:11.257 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:52:11.257 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:52:11.257 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:11.257 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:11.257 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:11.257 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-mon.a.asok 2026-02-18T08:52:11.257 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:52:11.258 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7523/ceph-mon.a.asok config get fsid 2026-02-18T08:52:11.313 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:52:11.313 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:52:11.313 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:52:11.313 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:52:11.313 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:52:11.314 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:52:11.314 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:52:11.314 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:52:11.314 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:11.315 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:11.315 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:11.315 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-mon.a.asok 2026-02-18T08:52:11.315 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:52:11.315 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7523/ceph-mon.a.asok config get mon_host 2026-02-18T08:52:11.369 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:206: TEST_recovery_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-18T08:52:11.369 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-18T08:52:11.369 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:52:11.369 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:52:11.369 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:52:11.370 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-18T08:52:11.370 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:52:11.478 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:52:11.478 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:52:11.478 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:52:11.478 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:52:11.479 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:11.479 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:11.479 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:52:11.479 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:52:11.480 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:52:11.500 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:208: TEST_recovery_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-18T08:52:11.500 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-18T08:52:11.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:52:11.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:52:11.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:52:11.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-18T08:52:11.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-18T08:52:11.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:52:11.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:52:11.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:52:11.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-18T08:52:11.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-18T08:52:11.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:52:11.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:52:11.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:52:11.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-18T08:52:11.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-18T08:52:11.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:52:11.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:52:11.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:52:11.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:52:11.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:52:11.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-18T08:52:11.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-18T08:52:11.503 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:52:11.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d285f0ec-fd22-4d80-9c75-0642c240899c 2026-02-18T08:52:11.504 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 d285f0ec-fd22-4d80-9c75-0642c240899c 2026-02-18T08:52:11.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d285f0ec-fd22-4d80-9c75-0642c240899c' 2026-02-18T08:52:11.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:52:11.512 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC7fZVpxOxFHhAA1J8xz41TA+Cum1s4ErDxHQ== 2026-02-18T08:52:11.512 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC7fZVpxOxFHhAA1J8xz41TA+Cum1s4ErDxHQ=="}' 2026-02-18T08:52:11.512 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d285f0ec-fd22-4d80-9c75-0642c240899c -i td/mclock-config/0/new.json 2026-02-18T08:52:11.620 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:52:11.627 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-18T08:52:11.629 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQC7fZVpxOxFHhAA1J8xz41TA+Cum1s4ErDxHQ== --osd-uuid d285f0ec-fd22-4d80-9c75-0642c240899c 2026-02-18T08:52:11.643 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:11.636+0000 7f4106ac5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:11.645 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:11.636+0000 7f4106ac5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:11.646 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:11.636+0000 7f4106ac5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:11.647 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:11.640+0000 7f4106ac5a40 -1 bdev(0x564e9d4ee400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:52:11.647 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:11.640+0000 7f4106ac5a40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-18T08:52:12.413 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-18T08:52:12.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:52:12.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:52:12.415 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:52:12.415 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:52:12.535 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:52:12.536 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:52:12.536 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=b3905741-1e09-42e0-b88f-9467373ce724 --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7523/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-18T08:52:12.536 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:52:12.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:52:12.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:52:12.558 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:12.548+0000 7ff6498aaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:12.560 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:12.552+0000 7ff6498aaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:12.561 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:12.552+0000 7ff6498aaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:12.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:52:12.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:52:12.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:52:12.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:52:12.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:52:12.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:12.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:52:12.647 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:52:12.647 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:12.647 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:52:12.755 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:52:12.948 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:12.940+0000 7ff6498aaa40 -1 Falling back to public interface 2026-02-18T08:52:13.584 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:13.576+0000 7ff6498aaa40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:52:13.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:52:13.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:13.757 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:52:13.758 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:52:13.758 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:13.758 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:52:13.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:52:14.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:52:14.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:14.865 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:52:14.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:52:14.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:14.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:52:15.080 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/578904832,v1:127.0.0.1:6801/578904832] [v2:127.0.0.1:6802/578904832,v1:127.0.0.1:6803/578904832] exists,up d285f0ec-fd22-4d80-9c75-0642c240899c 2026-02-18T08:52:15.081 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:52:15.081 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:52:15.081 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:52:15.081 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:52:15.081 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:52:15.081 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:52:15.082 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:15.082 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:15.082 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:15.082 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:52:15.083 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:52:15.083 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-18T08:52:15.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: local 'recoveries={"osd_recovery_max_active":"0"}' 2026-02-18T08:52:15.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: echo '{"osd_recovery_max_active":"0"}' 2026-02-18T08:52:15.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: grep --quiet osd_recovery_max_active 2026-02-18T08:52:15.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:217: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-18T08:52:15.361 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:218: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-18T08:52:17.364 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:52:17.364 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:52:17.364 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:52:17.365 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:17.365 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:17.366 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:17.366 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:52:17.366 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:52:17.366 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-18T08:52:17.424 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: local 'max_recoveries={"osd_recovery_max_active":"0"}' 2026-02-18T08:52:17.424 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:221: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"0"}' = '{"osd_recovery_max_active":"0"}' 2026-02-18T08:52:17.424 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:225: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-18T08:52:17.641 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:226: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-18T08:52:17.867 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:227: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-18T08:52:19.869 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-18T08:52:19.870 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:52:19.870 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:52:19.871 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:19.871 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:19.871 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:19.871 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7523/ceph-osd.0.asok 2026-02-18T08:52:19.871 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-18T08:52:19.871 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7523/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-18T08:52:19.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: max_recoveries='{"osd_recovery_max_active":"10"}' 2026-02-18T08:52:19.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:230: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"10"}' = '{"osd_recovery_max_active":"10"}' 2026-02-18T08:52:19.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:232: TEST_recovery_limit_adjustment_mclock: teardown td/mclock-config 2026-02-18T08:52:19.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:52:19.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:52:19.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:52:19.932 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:19.932 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:19.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:19.933 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:19.933 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:20.040 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:20.040 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:20.041 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:20.041 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:20.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:20.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:20.043 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:20.044 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:20.044 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:20.044 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:20.045 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:20.045 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:20.046 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:52:20.046 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:52:20.052 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:20.052 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:20.052 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:20.052 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:52:20.053 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:20.053 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:20.054 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-18T08:52:20.054 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:52:20.054 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:52:20.054 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:52:20.054 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:20.054 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:20.055 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:20.055 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:20.055 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:20.056 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:20.056 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:20.057 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:20.058 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:20.059 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:20.059 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:20.060 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:20.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:20.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:20.062 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:20.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:20.063 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:20.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:52:20.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:52:20.065 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:20.065 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:20.066 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:20.066 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:52:20.067 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:20.067 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:20.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-18T08:52:20.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/mclock-config 0 2026-02-18T08:52:20.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-18T08:52:20.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-18T08:52:20.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-18T08:52:20.068 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:20.068 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:20.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:20.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:20.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:20.069 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:20.070 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:20.070 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:20.071 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:20.072 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:20.072 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:20.072 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:20.073 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:20.074 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:20.074 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:20.074 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:20.075 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:20.076 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-18T08:52:20.076 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-18T08:52:20.077 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:20.078 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:20.078 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7523 2026-02-18T08:52:20.078 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7523 2026-02-18T08:52:20.079 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:20.079 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:20.079 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-18T08:52:20.080 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-18T08:52:20.080 DEBUG:teuthology.orchestra.run.trial118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-18T08:52:20.131 INFO:tasks.workunit:Running workunit misc/network-ping.sh... 2026-02-18T08:52:20.131 DEBUG:teuthology.orchestra.run.trial118:workunit test misc/network-ping.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh 2026-02-18T08:52:20.183 INFO:tasks.workunit.client.0.trial118.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-18T08:52:20.188 INFO:tasks.workunit.client.0.trial118.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-18T08:52:20.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-18T08:52:20.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-18T08:52:20.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-18T08:52:20.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-18T08:52:20.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-18T08:52:20.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-18T08:52:20.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-18T08:52:20.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-18T08:52:20.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/network-ping 2026-02-18T08:52:20.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:6: run: local dir=td/network-ping 2026-02-18T08:52:20.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:7: run: shift 2026-02-18T08:52:20.190 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: export CEPH_MON=127.0.0.1:7146 2026-02-18T08:52:20.190 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: CEPH_MON=127.0.0.1:7146 2026-02-18T08:52:20.190 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:10: run: export CEPH_ARGS 2026-02-18T08:52:20.190 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: uuidgen 2026-02-18T08:52:20.191 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: CEPH_ARGS+='--fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none ' 2026-02-18T08:52:20.191 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:12: run: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-18T08:52:20.191 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:13: run: CEPH_ARGS+='--debug_disable_randomized_ping=true ' 2026-02-18T08:52:20.191 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:14: run: CEPH_ARGS+='--debug_heartbeat_testing_span=5 ' 2026-02-18T08:52:20.191 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:15: run: CEPH_ARGS+='--osd_heartbeat_interval=1 ' 2026-02-18T08:52:20.191 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: set 2026-02-18T08:52:20.191 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-18T08:52:20.193 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: local 'funcs=TEST_network_ping_test1 2026-02-18T08:52:20.193 INFO:tasks.workunit.client.0.trial118.stderr:TEST_network_ping_test2' 2026-02-18T08:52:20.193 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-18T08:52:20.193 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-18T08:52:20.193 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-18T08:52:20.193 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-18T08:52:20.193 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-18T08:52:20.193 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:52:20.193 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-18T08:52:20.193 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:52:20.194 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:52:20.194 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:52:20.194 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:52:20.194 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:52:20.194 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:52:20.195 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:52:20.195 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:52:20.196 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:52:20.196 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:52:20.196 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:52:20.197 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:52:20.198 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:20.198 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:52:20.198 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:52:20.198 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:52:20.199 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:52:20.200 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:52:20.200 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-18T08:52:20.202 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:52:20.202 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:20.202 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:52:20.202 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19969 2026-02-18T08:52:20.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:52:20.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:52:20.204 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-18T08:52:20.205 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:52:20.205 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:20.205 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:52:20.205 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19969 2026-02-18T08:52:20.207 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:52:20.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-18T08:52:20.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-18T08:52:20.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:52:20.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-18T08:52:20.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test1 td/network-ping 2026-02-18T08:52:20.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:25: TEST_network_ping_test1: local dir=td/network-ping 2026-02-18T08:52:20.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:27: TEST_network_ping_test1: run_mon td/network-ping a 2026-02-18T08:52:20.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-18T08:52:20.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:52:20.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:52:20.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:52:20.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-18T08:52:20.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-18T08:52:20.232 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:52:20.233 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:52:20.233 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:52:20.233 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:52:20.233 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:20.233 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:52:20.233 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:52:20.233 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:52:20.263 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:52:20.264 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:52:20.264 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:52:20.264 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:52:20.264 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:52:20.265 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:52:20.265 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:52:20.265 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:52:20.266 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:52:20.266 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:20.266 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:20.266 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:52:20.266 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-mon.a.asok 2026-02-18T08:52:20.266 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:52:20.266 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19969/ceph-mon.a.asok config get fsid 2026-02-18T08:52:20.320 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:52:20.320 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:52:20.320 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:52:20.320 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:52:20.321 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:52:20.321 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:52:20.321 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:52:20.321 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:52:20.322 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:52:20.322 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:20.322 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:52:20.322 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-mon.a.asok 2026-02-18T08:52:20.322 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:52:20.322 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19969/ceph-mon.a.asok config get mon_host 2026-02-18T08:52:20.375 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:28: TEST_network_ping_test1: run_mgr td/network-ping x 2026-02-18T08:52:20.375 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-18T08:52:20.375 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:52:20.375 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:52:20.375 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:52:20.375 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-18T08:52:20.375 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:52:20.486 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:52:20.486 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:52:20.486 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:52:20.486 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:52:20.486 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:20.486 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:52:20.486 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:52:20.487 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:52:20.487 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:52:20.507 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:29: TEST_network_ping_test1: run_osd td/network-ping 0 2026-02-18T08:52:20.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-18T08:52:20.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:52:20.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:52:20.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:52:20.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-18T08:52:20.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-18T08:52:20.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:52:20.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:52:20.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:52:20.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:52:20.509 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:52:20.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-18T08:52:20.511 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:52:20.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=44542de8-9e09-4924-ae9f-8af0deefc002 2026-02-18T08:52:20.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 44542de8-9e09-4924-ae9f-8af0deefc002' 2026-02-18T08:52:20.512 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 44542de8-9e09-4924-ae9f-8af0deefc002 2026-02-18T08:52:20.512 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:52:20.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDEfZVpk/m/HhAAwOkQhXNtlRUFK5vmzyasYQ== 2026-02-18T08:52:20.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDEfZVpk/m/HhAAwOkQhXNtlRUFK5vmzyasYQ=="}' 2026-02-18T08:52:20.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 44542de8-9e09-4924-ae9f-8af0deefc002 -i td/network-ping/0/new.json 2026-02-18T08:52:20.633 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:52:20.640 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-18T08:52:20.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDEfZVpk/m/HhAAwOkQhXNtlRUFK5vmzyasYQ== --osd-uuid 44542de8-9e09-4924-ae9f-8af0deefc002 2026-02-18T08:52:20.658 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:20.648+0000 7f0348673a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:20.659 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:20.652+0000 7f0348673a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:20.660 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:20.652+0000 7f0348673a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:20.661 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:20.652+0000 7f0348673a40 -1 bdev(0x562ffdb5e400 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:52:20.661 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:20.652+0000 7f0348673a40 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-18T08:52:21.329 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-18T08:52:21.330 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:52:21.330 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:52:21.331 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:52:21.331 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:52:21.445 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:52:21.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:52:21.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:52:21.447 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:52:21.447 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:52:21.447 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:52:21.461 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:21.452+0000 7fcb3c7a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:21.463 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:21.456+0000 7fcb3c7a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:21.463 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:21.456+0000 7fcb3c7a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:21.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:52:21.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:52:21.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:52:21.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:52:21.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:52:21.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:21.555 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:52:21.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:52:21.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:21.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:52:21.665 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:52:21.884 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:21.876+0000 7fcb3c7a4a40 -1 Falling back to public interface 2026-02-18T08:52:22.555 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:22.548+0000 7fcb3c7a4a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:52:22.666 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:52:22.666 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:22.667 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:52:22.667 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:52:22.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:22.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:52:22.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:52:23.789 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:52:23.789 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:23.789 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:52:23.790 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:52:23.790 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:23.790 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:52:23.999 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/436608896,v1:127.0.0.1:6801/436608896] [v2:127.0.0.1:6802/436608896,v1:127.0.0.1:6803/436608896] exists,up 44542de8-9e09-4924-ae9f-8af0deefc002 2026-02-18T08:52:24.002 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:52:24.002 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:52:24.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:52:24.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:30: TEST_network_ping_test1: run_osd td/network-ping 1 2026-02-18T08:52:24.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-18T08:52:24.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:52:24.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-18T08:52:24.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:52:24.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-18T08:52:24.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-18T08:52:24.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:52:24.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:52:24.004 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:52:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-18T08:52:24.006 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:52:24.006 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=16bcdc4a-763d-498f-835e-23fd5f58477e 2026-02-18T08:52:24.006 INFO:tasks.workunit.client.0.trial118.stdout:add osd1 16bcdc4a-763d-498f-835e-23fd5f58477e 2026-02-18T08:52:24.007 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 16bcdc4a-763d-498f-835e-23fd5f58477e' 2026-02-18T08:52:24.007 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:52:24.016 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDIfZVpz9G1ABAAZf/uFyLkgQ7PZYMQ6BSL1w== 2026-02-18T08:52:24.016 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDIfZVpz9G1ABAAZf/uFyLkgQ7PZYMQ6BSL1w=="}' 2026-02-18T08:52:24.016 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 16bcdc4a-763d-498f-835e-23fd5f58477e -i td/network-ping/1/new.json 2026-02-18T08:52:24.237 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:52:24.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-18T08:52:24.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDIfZVpz9G1ABAAZf/uFyLkgQ7PZYMQ6BSL1w== --osd-uuid 16bcdc4a-763d-498f-835e-23fd5f58477e 2026-02-18T08:52:24.273 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:24.264+0000 7fc0b3251a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:24.274 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:24.268+0000 7fc0b3251a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:24.276 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:24.268+0000 7fc0b3251a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:24.276 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:24.268+0000 7fc0b3251a40 -1 bdev(0x55b51fb36400 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-18T08:52:24.276 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:24.268+0000 7fc0b3251a40 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-18T08:52:24.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-18T08:52:24.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:52:24.939 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-18T08:52:24.939 INFO:tasks.workunit.client.0.trial118.stdout:adding osd1 key to auth repository 2026-02-18T08:52:24.939 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:52:25.234 INFO:tasks.workunit.client.0.trial118.stdout:start osd.1 2026-02-18T08:52:25.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-18T08:52:25.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:52:25.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:52:25.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:52:25.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:52:25.258 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:25.252+0000 7f7e9b85ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:25.260 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:25.252+0000 7f7e9b85ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:25.261 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:25.252+0000 7f7e9b85ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:25.454 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-18T08:52:25.454 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:52:25.454 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-18T08:52:25.455 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:52:25.455 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:52:25.455 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:25.455 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:52:25.456 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:52:25.456 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:25.456 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:52:25.666 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:52:25.712 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:25.704+0000 7f7e9b85ea40 -1 Falling back to public interface 2026-02-18T08:52:26.336 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:26.328+0000 7f7e9b85ea40 -1 osd.1 0 log_to_monitors true 2026-02-18T08:52:26.667 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:52:26.667 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:26.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:52:26.668 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:52:26.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:26.669 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:52:26.881 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:52:27.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:52:27.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:27.883 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:52:27.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:52:27.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:27.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:52:28.100 INFO:tasks.workunit.client.0.trial118.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3477445166,v1:127.0.0.1:6811/3477445166] [v2:127.0.0.1:6812/3477445166,v1:127.0.0.1:6813/3477445166] exists,up 16bcdc4a-763d-498f-835e-23fd5f58477e 2026-02-18T08:52:28.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:52:28.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:52:28.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:52:28.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:31: TEST_network_ping_test1: run_osd td/network-ping 2 2026-02-18T08:52:28.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-18T08:52:28.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:52:28.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-18T08:52:28.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:52:28.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-18T08:52:28.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:52:28.105 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:52:28.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:52:28.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:52:28.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:52:28.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-18T08:52:28.107 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:52:28.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=55acb6dc-8aba-4462-b90e-fae8aafa2301 2026-02-18T08:52:28.108 INFO:tasks.workunit.client.0.trial118.stdout:add osd2 55acb6dc-8aba-4462-b90e-fae8aafa2301 2026-02-18T08:52:28.108 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 55acb6dc-8aba-4462-b90e-fae8aafa2301' 2026-02-18T08:52:28.108 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:52:28.117 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDMfZVpaNO1BhAAjq7NAd9B4EEhbKpsMXegXw== 2026-02-18T08:52:28.117 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDMfZVpaNO1BhAAjq7NAd9B4EEhbKpsMXegXw=="}' 2026-02-18T08:52:28.117 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 55acb6dc-8aba-4462-b90e-fae8aafa2301 -i td/network-ping/2/new.json 2026-02-18T08:52:28.341 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:52:28.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-18T08:52:28.353 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDMfZVpaNO1BhAAjq7NAd9B4EEhbKpsMXegXw== --osd-uuid 55acb6dc-8aba-4462-b90e-fae8aafa2301 2026-02-18T08:52:28.377 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:28.368+0000 7ff816e0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:28.378 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:28.372+0000 7ff816e0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:28.379 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:28.372+0000 7ff816e0ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:28.380 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:28.372+0000 7ff816e0ca40 -1 bdev(0x562e496b2400 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-18T08:52:28.380 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:28.372+0000 7ff816e0ca40 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-18T08:52:28.998 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-18T08:52:28.998 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:52:28.999 INFO:tasks.workunit.client.0.trial118.stdout:adding osd2 key to auth repository 2026-02-18T08:52:28.999 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-18T08:52:29.000 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:52:29.285 INFO:tasks.workunit.client.0.trial118.stdout:start osd.2 2026-02-18T08:52:29.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-18T08:52:29.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:52:29.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:52:29.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:52:29.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:52:29.309 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:29.300+0000 7f506b25fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:29.311 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:29.304+0000 7f506b25fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:29.312 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:29.304+0000 7f506b25fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:52:29.498 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-18T08:52:29.498 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:52:29.498 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-18T08:52:29.499 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:52:29.499 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:52:29.499 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:29.499 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:52:29.500 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:52:29.500 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:29.500 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-18T08:52:29.708 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:52:29.772 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:29.764+0000 7f506b25fa40 -1 Falling back to public interface 2026-02-18T08:52:30.465 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:52:30.456+0000 7f506b25fa40 -1 osd.2 0 log_to_monitors true 2026-02-18T08:52:30.710 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:52:30.710 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:30.710 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:52:30.710 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:52:30.711 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:30.711 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-18T08:52:30.917 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:52:31.919 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:52:31.919 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:52:31.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:52:31.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:52:31.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:52:31.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-18T08:52:32.129 INFO:tasks.workunit.client.0.trial118.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/799101997,v1:127.0.0.1:6819/799101997] [v2:127.0.0.1:6820/799101997,v1:127.0.0.1:6821/799101997] exists,up 55acb6dc-8aba-4462-b90e-fae8aafa2301 2026-02-18T08:52:32.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:52:32.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:52:32.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:52:32.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:33: TEST_network_ping_test1: sleep 5 2026-02-18T08:52:37.130 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:35: TEST_network_ping_test1: create_pool foo 16 2026-02-18T08:52:37.131 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-18T08:52:37.390 INFO:tasks.workunit.client.0.trial118.stderr:pool 'foo' already exists 2026-02-18T08:52:37.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T08:52:38.401 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:38: TEST_network_ping_test1: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-18T08:52:38.422 INFO:tasks.workunit.client.0.trial118.stdout:hints = 1 2026-02-18T08:52:38.422 INFO:tasks.workunit.client.0.trial118.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-18T08:52:38.422 INFO:tasks.workunit.client.0.trial118.stdout:Object prefix: benchmark_data_trial118_25678 2026-02-18T08:52:38.422 INFO:tasks.workunit.client.0.trial118.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-18T08:52:38.423 INFO:tasks.workunit.client.0.trial118.stdout: 0 0 0 0 0 0 - 0 2026-02-18T08:52:39.422 INFO:tasks.workunit.client.0.trial118.stdout: 1 16 10605 10589 41.3595 41.3633 0.00143914 0.00150883 2026-02-18T08:52:40.422 INFO:tasks.workunit.client.0.trial118.stdout: 2 16 21436 21420 41.8324 42.3086 0.000648069 0.00149155 2026-02-18T08:52:41.422 INFO:tasks.workunit.client.0.trial118.stdout: 3 16 32719 32703 42.5786 44.0742 0.000938526 0.00146613 2026-02-18T08:52:42.422 INFO:tasks.workunit.client.0.trial118.stdout: 4 16 43976 43960 42.9262 43.9727 0.00184246 0.00145438 2026-02-18T08:52:43.423 INFO:tasks.workunit.client.0.trial118.stdout: 5 16 54897 54881 42.8724 42.6602 0.00171437 0.00145623 2026-02-18T08:52:44.423 INFO:tasks.workunit.client.0.trial118.stdout: 6 16 66436 66420 43.2387 45.0742 0.000900656 0.00144385 2026-02-18T08:52:45.423 INFO:tasks.workunit.client.0.trial118.stdout: 7 16 77192 77176 43.0636 42.0156 0.00126136 0.00144978 2026-02-18T08:52:46.423 INFO:tasks.workunit.client.0.trial118.stdout: 8 16 87275 87259 42.6036 39.3867 0.00157257 0.00146546 2026-02-18T08:52:47.423 INFO:tasks.workunit.client.0.trial118.stdout: 9 16 96740 96724 41.9776 36.9727 0.00167362 0.00148735 2026-02-18T08:52:48.423 INFO:tasks.workunit.client.0.trial118.stdout: 10 5 106114 106109 41.4456 36.6602 0.0013511 0.00150653 2026-02-18T08:52:49.423 INFO:tasks.workunit.client.0.trial118.stdout:Total time run: 10.0012 2026-02-18T08:52:49.424 INFO:tasks.workunit.client.0.trial118.stdout:Total writes made: 106114 2026-02-18T08:52:49.424 INFO:tasks.workunit.client.0.trial118.stdout:Write size: 4096 2026-02-18T08:52:49.424 INFO:tasks.workunit.client.0.trial118.stdout:Object size: 4096 2026-02-18T08:52:49.424 INFO:tasks.workunit.client.0.trial118.stdout:Bandwidth (MB/sec): 41.4458 2026-02-18T08:52:49.424 INFO:tasks.workunit.client.0.trial118.stdout:Stddev Bandwidth: 2.909 2026-02-18T08:52:49.424 INFO:tasks.workunit.client.0.trial118.stdout:Max bandwidth (MB/sec): 45.0742 2026-02-18T08:52:49.424 INFO:tasks.workunit.client.0.trial118.stdout:Min bandwidth (MB/sec): 36.6602 2026-02-18T08:52:49.424 INFO:tasks.workunit.client.0.trial118.stdout:Average IOPS: 10610 2026-02-18T08:52:49.424 INFO:tasks.workunit.client.0.trial118.stdout:Stddev IOPS: 744.705 2026-02-18T08:52:49.424 INFO:tasks.workunit.client.0.trial118.stdout:Max IOPS: 11539 2026-02-18T08:52:49.425 INFO:tasks.workunit.client.0.trial118.stdout:Min IOPS: 9385 2026-02-18T08:52:49.425 INFO:tasks.workunit.client.0.trial118.stdout:Average Latency(s): 0.00150651 2026-02-18T08:52:49.425 INFO:tasks.workunit.client.0.trial118.stdout:Stddev Latency(s): 0.000692172 2026-02-18T08:52:49.425 INFO:tasks.workunit.client.0.trial118.stdout:Max latency(s): 0.00934185 2026-02-18T08:52:49.425 INFO:tasks.workunit.client.0.trial118.stdout:Min latency(s): 0.000441146 2026-02-18T08:52:49.429 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:41: TEST_network_ping_test1: sleep 10 2026-02-18T08:52:59.431 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:42: TEST_network_ping_test1: flush_pg_stats 2026-02-18T08:52:59.431 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:52:59.431 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:53:00.300 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-18T08:53:00.300 INFO:tasks.workunit.client.0.trial118.stderr:1 2026-02-18T08:53:00.300 INFO:tasks.workunit.client.0.trial118.stderr:2' 2026-02-18T08:53:00.301 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:53:00.301 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:53:00.301 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:53:00.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-18T08:53:00.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-18T08:53:00.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-18T08:53:00.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:53:00.884 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-18T08:53:01.202 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705673 2026-02-18T08:53:01.202 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705673 2026-02-18T08:53:01.202 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673' 2026-02-18T08:53:01.202 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:53:01.202 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-18T08:53:01.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574856 2026-02-18T08:53:01.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574856 2026-02-18T08:53:01.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-38654705673 2-55834574856' 2026-02-18T08:53:01.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:53:01.280 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-18T08:53:01.281 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:53:01.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:53:01.282 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-18T08:53:01.282 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:53:01.284 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-18T08:53:01.284 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-18T08:53:01.284 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836490 2026-02-18T08:53:01.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:53:01.499 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-18T08:53:01.500 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:53:02.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:53:02.502 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:53:02.713 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-18T08:53:02.714 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:53:03.715 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-18T08:53:03.716 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:53:03.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836490 -lt 21474836490 2026-02-18T08:53:03.933 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:53:03.933 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705673 2026-02-18T08:53:03.933 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:53:03.934 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-18T08:53:03.935 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705673 2026-02-18T08:53:03.935 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:53:03.937 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705673 2026-02-18T08:53:03.937 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.1 seq 38654705673 2026-02-18T08:53:03.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705673' 2026-02-18T08:53:03.938 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T08:53:04.152 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705674 -lt 38654705673 2026-02-18T08:53:04.152 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:53:04.152 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574856 2026-02-18T08:53:04.153 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:53:04.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-18T08:53:04.154 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574856 2026-02-18T08:53:04.154 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:53:04.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574856 2026-02-18T08:53:04.156 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.2 seq 55834574856 2026-02-18T08:53:04.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574856' 2026-02-18T08:53:04.156 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-18T08:53:04.376 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574857 -lt 55834574856 2026-02-18T08:53:04.376 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:53:04.377 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: get_asok_path osd.0 2026-02-18T08:53:04.377 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:53:04.377 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:53:04.377 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:53:04.377 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:53:04.378 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:53:04.378 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-osd.0.asok 2026-02-18T08:53:04.378 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:53:04.378 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-osd.0.asok dump_osd_network 2026-02-18T08:53:04.432 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:53:04.432 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 1000, 2026-02-18T08:53:04.432 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [] 2026-02-18T08:53:04.432 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:53:04.439 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:04.439 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:53:04.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:53:04.451 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:04.451 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: jq .threshold 2026-02-18T08:53:04.462 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: test 1000 = 1000 2026-02-18T08:53:04.462 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:53:04.463 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-18T08:53:04.463 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-18T08:53:04.463 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-18T08:53:04.464 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:53:04.464 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:53:04.464 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:53:04.465 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-mgr.x.asok 2026-02-18T08:53:04.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:53:04.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-mgr.x.asok dump_osd_network 2026-02-18T08:53:04.511 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:53:04.511 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 1000, 2026-02-18T08:53:04.511 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [] 2026-02-18T08:53:04.512 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:53:04.518 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:04.518 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:53:04.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:53:04.529 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:04.529 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: jq .threshold 2026-02-18T08:53:04.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: test 1000 = 1000 2026-02-18T08:53:04.539 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:53:04.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: get_asok_path osd.0 2026-02-18T08:53:04.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:53:04.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:53:04.541 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:53:04.541 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:53:04.541 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:53:04.541 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-osd.0.asok 2026-02-18T08:53:04.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:53:04.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-osd.0.asok dump_osd_network 0 2026-02-18T08:53:04.592 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:53:04.592 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 0, 2026-02-18T08:53:04.592 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [ 2026-02-18T08:53:04.593 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.593 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:03 2026", 2026-02-18T08:53:04.593 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.593 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:04.593 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:04.593 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:04.593 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.593 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.157, 2026-02-18T08:53:04.593 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.190, 2026-02-18T08:53:04.593 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.201 2026-02-18T08:53:04.593 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.202, 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.411, 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.545 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.202, 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.411, 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.545 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.157 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.594 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:01 2026", 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.162, 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.196, 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.198 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.196, 2026-02-18T08:53:04.595 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.638, 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.638 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.196, 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.638, 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.638 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.167 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:01 2026", 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.596 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.168, 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.178, 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.188 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.189, 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.345, 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.345 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.597 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.189, 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.345, 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.345 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.179 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:03 2026", 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:04.598 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.138, 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.151, 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.158 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.151, 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.235, 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.288 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.151, 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.235, 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.288 2026-02-18T08:53:04.599 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.600 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.144 2026-02-18T08:53:04.600 INFO:tasks.workunit.client.0.trial118.stdout: } 2026-02-18T08:53:04.600 INFO:tasks.workunit.client.0.trial118.stdout: ] 2026-02-18T08:53:04.600 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:53:04.601 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:04.601 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:53:04.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: test 4 = 4 2026-02-18T08:53:04.608 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:04.609 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: jq .threshold 2026-02-18T08:53:04.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:53:04.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:53:04.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-18T08:53:04.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-18T08:53:04.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-18T08:53:04.620 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:53:04.620 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:53:04.620 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:53:04.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-mgr.x.asok 2026-02-18T08:53:04.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:53:04.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-mgr.x.asok dump_osd_network 0 2026-02-18T08:53:04.666 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:53:04.666 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 0, 2026-02-18T08:53:04.666 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [ 2026-02-18T08:53:04.666 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.667 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:02 2026", 2026-02-18T08:53:04.667 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.667 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:53:04.667 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:04.667 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:04.667 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.667 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.188, 2026-02-18T08:53:04.667 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.311, 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.245 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.140, 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.137, 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.137 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.236, 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 1.822, 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 1.822 2026-02-18T08:53:04.668 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.669 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.207 2026-02-18T08:53:04.669 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.669 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.669 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:02 2026", 2026-02-18T08:53:04.669 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.669 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:53:04.669 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:53:04.669 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:04.669 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.669 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.171, 2026-02-18T08:53:04.669 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.220, 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.271 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.137, 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.128, 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.128 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.222, 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 1.412, 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 1.412 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.670 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.137 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:02 2026", 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.205, 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.237, 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.235 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.671 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.184, 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.096, 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.096 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.224, 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 1.280, 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 1.280 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.220 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.672 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:02 2026", 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.175, 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.194, 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.227 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.152, 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.103, 2026-02-18T08:53:04.673 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.103 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.191, 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 1.435, 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 1.435 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.185 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:52:58 2026", 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:04.674 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.133, 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.207, 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.204 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.108, 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.108, 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.108 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.155, 2026-02-18T08:53:04.675 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.545, 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.545 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.146 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:52:56 2026", 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.149, 2026-02-18T08:53:04.676 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.203, 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.201 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.141, 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.110, 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.110 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.157, 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.638, 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.638 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.677 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.177 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:52:56 2026", 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.134, 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.183, 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.190 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.678 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.114, 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.106, 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.106 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.150, 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.345, 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.345 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.165 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.679 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:01 2026", 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.158, 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.164, 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.189 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.138, 2026-02-18T08:53:04.680 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.115, 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.115 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.204, 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.266, 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.266 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.164 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:01 2026", 2026-02-18T08:53:04.681 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.682 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:53:04.682 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:04.682 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:04.682 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.682 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.171, 2026-02-18T08:53:04.682 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.172, 2026-02-18T08:53:04.682 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.188 2026-02-18T08:53:04.682 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.682 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.682 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.150, 2026-02-18T08:53:04.682 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.140, 2026-02-18T08:53:04.683 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.140 2026-02-18T08:53:04.683 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.683 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.683 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.190, 2026-02-18T08:53:04.683 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.226, 2026-02-18T08:53:04.683 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.282 2026-02-18T08:53:04.683 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.683 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.174 2026-02-18T08:53:04.683 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.683 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.683 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:01 2026", 2026-02-18T08:53:04.684 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.684 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:53:04.684 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:53:04.684 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:04.684 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.684 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.137, 2026-02-18T08:53:04.684 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.184, 2026-02-18T08:53:04.684 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.186 2026-02-18T08:53:04.684 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.684 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.684 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.116, 2026-02-18T08:53:04.685 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.112, 2026-02-18T08:53:04.685 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.112 2026-02-18T08:53:04.685 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.685 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.685 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.171, 2026-02-18T08:53:04.685 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.572, 2026-02-18T08:53:04.685 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.572 2026-02-18T08:53:04.685 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.685 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.116 2026-02-18T08:53:04.685 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:52:58 2026", 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.131, 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.155, 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.160 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.686 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.115, 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.111, 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.090 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.160, 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.235, 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.288 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.109 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:01 2026", 2026-02-18T08:53:04.687 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.132, 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.156, 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.150 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.107, 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.100, 2026-02-18T08:53:04.688 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.100 2026-02-18T08:53:04.689 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.689 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:04.689 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.153, 2026-02-18T08:53:04.689 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.305, 2026-02-18T08:53:04.689 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.305 2026-02-18T08:53:04.689 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:04.689 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.146 2026-02-18T08:53:04.689 INFO:tasks.workunit.client.0.trial118.stdout: } 2026-02-18T08:53:04.689 INFO:tasks.workunit.client.0.trial118.stdout: ] 2026-02-18T08:53:04.689 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:53:04.691 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:04.691 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:53:04.691 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: test 12 = 12 2026-02-18T08:53:04.691 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:04.691 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: jq .threshold 2026-02-18T08:53:04.694 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:53:04.694 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:61: TEST_network_ping_test1: sleep 20 2026-02-18T08:53:24.695 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:62: TEST_network_ping_test1: flush_pg_stats 2026-02-18T08:53:24.696 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:53:24.696 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:53:24.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-18T08:53:24.910 INFO:tasks.workunit.client.0.trial118.stderr:1 2026-02-18T08:53:24.910 INFO:tasks.workunit.client.0.trial118.stderr:2' 2026-02-18T08:53:24.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:53:24.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:53:24.911 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:53:24.990 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836497 2026-02-18T08:53:24.991 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836497 2026-02-18T08:53:24.991 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497' 2026-02-18T08:53:24.991 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:53:24.991 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-18T08:53:25.072 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705680 2026-02-18T08:53:25.072 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705680 2026-02-18T08:53:25.072 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680' 2026-02-18T08:53:25.073 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:53:25.073 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-18T08:53:25.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574863 2026-02-18T08:53:25.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574863 2026-02-18T08:53:25.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836497 1-38654705680 2-55834574863' 2026-02-18T08:53:25.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:53:25.154 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836497 2026-02-18T08:53:25.154 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:53:25.155 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:53:25.155 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836497 2026-02-18T08:53:25.156 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:53:25.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836497 2026-02-18T08:53:25.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836497' 2026-02-18T08:53:25.157 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836497 2026-02-18T08:53:25.158 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:53:25.379 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836495 -lt 21474836497 2026-02-18T08:53:25.379 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:53:26.380 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:53:26.380 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:53:26.597 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836495 -lt 21474836497 2026-02-18T08:53:26.597 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:53:27.598 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-18T08:53:27.599 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:53:27.814 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836497 -lt 21474836497 2026-02-18T08:53:27.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:53:27.815 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705680 2026-02-18T08:53:27.816 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:53:27.816 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-18T08:53:27.817 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705680 2026-02-18T08:53:27.818 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:53:27.819 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705680 2026-02-18T08:53:27.819 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.1 seq 38654705680 2026-02-18T08:53:27.820 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705680' 2026-02-18T08:53:27.820 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T08:53:28.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705680 -lt 38654705680 2026-02-18T08:53:28.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:53:28.029 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574863 2026-02-18T08:53:28.029 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:53:28.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-18T08:53:28.031 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574863 2026-02-18T08:53:28.031 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:53:28.032 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574863 2026-02-18T08:53:28.032 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.2 seq 55834574863 2026-02-18T08:53:28.033 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574863' 2026-02-18T08:53:28.033 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-18T08:53:28.257 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574864 -lt 55834574863 2026-02-18T08:53:28.258 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:53:28.259 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: get_asok_path osd.0 2026-02-18T08:53:28.259 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:53:28.259 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:53:28.259 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:53:28.259 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:53:28.259 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:53:28.259 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-osd.0.asok 2026-02-18T08:53:28.260 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:53:28.260 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-osd.0.asok dump_osd_network 2026-02-18T08:53:28.312 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:53:28.313 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 1000, 2026-02-18T08:53:28.313 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [] 2026-02-18T08:53:28.313 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:53:28.319 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:28.320 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:53:28.330 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:53:28.331 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:28.331 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: jq .threshold 2026-02-18T08:53:28.340 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: test 1000 = 1000 2026-02-18T08:53:28.340 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:53:28.341 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-18T08:53:28.341 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-18T08:53:28.342 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-18T08:53:28.342 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:53:28.342 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:53:28.342 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:53:28.342 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-mgr.x.asok 2026-02-18T08:53:28.343 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:53:28.343 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-mgr.x.asok dump_osd_network 2026-02-18T08:53:28.389 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:53:28.389 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 1000, 2026-02-18T08:53:28.389 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [] 2026-02-18T08:53:28.389 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:53:28.396 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:28.396 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:53:28.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:53:28.407 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:28.407 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: jq .threshold 2026-02-18T08:53:28.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: test 1000 = 1000 2026-02-18T08:53:28.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:53:28.417 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: get_asok_path osd.0 2026-02-18T08:53:28.417 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:53:28.417 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:53:28.417 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:53:28.418 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:53:28.418 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:53:28.418 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-osd.0.asok 2026-02-18T08:53:28.418 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:53:28.418 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-osd.0.asok dump_osd_network 0 2026-02-18T08:53:28.469 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:53:28.469 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 0, 2026-02-18T08:53:28.469 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [ 2026-02-18T08:53:28.469 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:23 2026", 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.175, 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.162, 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.190 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.470 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.212, 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.212, 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.545 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.212, 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.212, 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.545 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.227 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.471 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:26 2026", 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.181, 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.169, 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.188 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.244, 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.244, 2026-02-18T08:53:28.472 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.638 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.244, 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.244, 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.638 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.168 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:26 2026", 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:28.473 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.178, 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.179, 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.183 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.229, 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.229, 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.345 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.474 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.229, 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.229, 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.345 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.184 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:23 2026", 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.475 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.143, 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.150, 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.156 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.191, 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.217, 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.288 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.191, 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.217, 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.288 2026-02-18T08:53:28.476 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.477 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.212 2026-02-18T08:53:28.477 INFO:tasks.workunit.client.0.trial118.stdout: } 2026-02-18T08:53:28.477 INFO:tasks.workunit.client.0.trial118.stdout: ] 2026-02-18T08:53:28.477 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:53:28.477 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:28.477 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:53:28.486 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: test 4 = 4 2026-02-18T08:53:28.487 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:28.487 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: jq .threshold 2026-02-18T08:53:28.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:53:28.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:53:28.497 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-18T08:53:28.497 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-18T08:53:28.498 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-18T08:53:28.498 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:53:28.498 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:53:28.498 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:53:28.498 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-mgr.x.asok 2026-02-18T08:53:28.498 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:53:28.499 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-mgr.x.asok dump_osd_network 0 2026-02-18T08:53:28.548 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:53:28.548 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 0, 2026-02-18T08:53:28.548 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [ 2026-02-18T08:53:28.548 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.548 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:22 2026", 2026-02-18T08:53:28.548 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.161, 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.179, 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.235 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.137, 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.128, 2026-02-18T08:53:28.549 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.128 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.203, 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.310, 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 1.822 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.166 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:22 2026", 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.550 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.159, 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.171, 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.234 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.114, 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.114, 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.114 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.551 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.187, 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.250, 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 1.412 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.130 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:22 2026", 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.552 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.194, 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.198, 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.225 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.158, 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.125, 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.096 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.210, 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.362, 2026-02-18T08:53:28.553 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 1.280 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.221 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:22 2026", 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.150, 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.163, 2026-02-18T08:53:28.554 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.203 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.102, 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.102, 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.102 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.175, 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.303, 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 1.435 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.185 2026-02-18T08:53:28.555 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:23 2026", 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.175, 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.162, 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.190 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.137, 2026-02-18T08:53:28.556 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.113, 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.108 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.212, 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.212, 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.545 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.097 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:21 2026", 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.557 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.157, 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.166, 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.189 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.146, 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.117, 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.110 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.558 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.182, 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.233, 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.638 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.202 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:21 2026", 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:28.559 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.166, 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.177, 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.185 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.126, 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.126, 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.106 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.184, 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.217, 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.345 2026-02-18T08:53:28.560 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.192 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:26 2026", 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.160, 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.166, 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.178 2026-02-18T08:53:28.561 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.149, 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.139, 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.139 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.169, 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.218, 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.282 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.169 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.562 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:26 2026", 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.155, 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.169, 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.178 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.125, 2026-02-18T08:53:28.563 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.115, 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.115 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.185, 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.222, 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.266 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.155 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:26 2026", 2026-02-18T08:53:28.564 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.156, 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.161, 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.177 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.127, 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.106, 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.106 2026-02-18T08:53:28.565 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.193, 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.202, 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.572 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.146 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:26 2026", 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:53:28.566 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.167, 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.141, 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.148 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.111, 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.110, 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.100 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.205, 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.205, 2026-02-18T08:53:28.567 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.305 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.197 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:53:23 2026", 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.143, 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.150, 2026-02-18T08:53:28.568 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.156 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.093, 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.093, 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.090 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.191, 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.217, 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.288 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.176 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: } 2026-02-18T08:53:28.569 INFO:tasks.workunit.client.0.trial118.stdout: ] 2026-02-18T08:53:28.570 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:53:28.571 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:28.571 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:53:28.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: test 12 = 12 2026-02-18T08:53:28.571 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:53:28.571 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: jq .threshold 2026-02-18T08:53:28.575 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:53:28.575 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:81: TEST_network_ping_test1: sleep 50 2026-02-18T08:54:18.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:82: TEST_network_ping_test1: flush_pg_stats 2026-02-18T08:54:18.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:54:18.579 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:54:18.796 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-18T08:54:18.797 INFO:tasks.workunit.client.0.trial118.stderr:1 2026-02-18T08:54:18.797 INFO:tasks.workunit.client.0.trial118.stderr:2' 2026-02-18T08:54:18.797 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:54:18.797 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:54:18.797 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:54:18.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836510 2026-02-18T08:54:18.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836510 2026-02-18T08:54:18.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836510' 2026-02-18T08:54:18.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:54:18.877 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-18T08:54:18.960 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705693 2026-02-18T08:54:18.960 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705693 2026-02-18T08:54:18.960 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836510 1-38654705693' 2026-02-18T08:54:18.960 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:54:18.960 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-18T08:54:19.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574876 2026-02-18T08:54:19.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574876 2026-02-18T08:54:19.039 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836510 1-38654705693 2-55834574876' 2026-02-18T08:54:19.039 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:54:19.039 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836510 2026-02-18T08:54:19.039 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:54:19.040 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:54:19.041 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836510 2026-02-18T08:54:19.041 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:54:19.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836510 2026-02-18T08:54:19.042 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836510 2026-02-18T08:54:19.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836510' 2026-02-18T08:54:19.043 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:54:19.261 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836507 -lt 21474836510 2026-02-18T08:54:19.261 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:54:20.262 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:54:20.262 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:54:20.475 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836507 -lt 21474836510 2026-02-18T08:54:20.475 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:54:21.476 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-18T08:54:21.477 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:54:21.694 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836510 -lt 21474836510 2026-02-18T08:54:21.694 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:54:21.695 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705693 2026-02-18T08:54:21.695 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:54:21.696 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-18T08:54:21.697 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705693 2026-02-18T08:54:21.697 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:54:21.698 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705693 2026-02-18T08:54:21.699 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.1 seq 38654705693 2026-02-18T08:54:21.699 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705693' 2026-02-18T08:54:21.699 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T08:54:21.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705693 -lt 38654705693 2026-02-18T08:54:21.909 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:54:21.909 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574876 2026-02-18T08:54:21.909 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:54:21.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-18T08:54:21.911 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574876 2026-02-18T08:54:21.911 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:54:21.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574876 2026-02-18T08:54:21.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574876' 2026-02-18T08:54:21.913 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.2 seq 55834574876 2026-02-18T08:54:21.913 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-18T08:54:22.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574876 -lt 55834574876 2026-02-18T08:54:22.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:54:22.139 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: get_asok_path osd.0 2026-02-18T08:54:22.139 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:54:22.139 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:54:22.139 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:54:22.139 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:22.139 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:22.140 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-osd.0.asok 2026-02-18T08:54:22.140 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:54:22.140 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-osd.0.asok dump_osd_network 2026-02-18T08:54:22.195 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:54:22.195 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 1000, 2026-02-18T08:54:22.195 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [] 2026-02-18T08:54:22.195 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:54:22.202 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:54:22.202 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:54:22.212 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:54:22.213 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:54:22.213 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: jq .threshold 2026-02-18T08:54:22.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: test 1000 = 1000 2026-02-18T08:54:22.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:54:22.225 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-18T08:54:22.225 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-18T08:54:22.225 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-18T08:54:22.225 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:54:22.225 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:22.225 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:22.226 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-mgr.x.asok 2026-02-18T08:54:22.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:54:22.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-mgr.x.asok dump_osd_network 2026-02-18T08:54:22.277 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:54:22.277 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 1000, 2026-02-18T08:54:22.277 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [] 2026-02-18T08:54:22.277 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:54:22.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:54:22.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:54:22.294 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:54:22.295 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:54:22.295 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: jq .threshold 2026-02-18T08:54:22.305 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: test 1000 = 1000 2026-02-18T08:54:22.305 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:54:22.306 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: get_asok_path osd.0 2026-02-18T08:54:22.307 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:54:22.307 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:54:22.307 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:54:22.307 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:22.307 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:22.307 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-osd.0.asok 2026-02-18T08:54:22.307 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:54:22.308 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-osd.0.asok dump_osd_network 0 2026-02-18T08:54:22.361 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:54:22.362 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 0, 2026-02-18T08:54:22.362 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [ 2026-02-18T08:54:22.362 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.362 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:21 2026", 2026-02-18T08:54:22.362 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.362 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:54:22.362 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:54:22.362 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:54:22.362 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.362 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.140, 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.149, 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.168 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.193, 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.193, 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.266 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.193, 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.193, 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.266 2026-02-18T08:54:22.363 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.193 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:21 2026", 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.136, 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.142, 2026-02-18T08:54:22.364 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.158 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.164, 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.190, 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.297 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.164, 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.190, 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.297 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.164 2026-02-18T08:54:22.365 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:18 2026", 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.127, 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.145, 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.155 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.366 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.144, 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.198, 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.248 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.144, 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.198, 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.248 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.176 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:18 2026", 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.367 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.104, 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.133, 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.148 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.111, 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.192, 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.233 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.368 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.369 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.111, 2026-02-18T08:54:22.369 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.192, 2026-02-18T08:54:22.369 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.233 2026-02-18T08:54:22.369 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.369 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.142 2026-02-18T08:54:22.369 INFO:tasks.workunit.client.0.trial118.stdout: } 2026-02-18T08:54:22.369 INFO:tasks.workunit.client.0.trial118.stdout: ] 2026-02-18T08:54:22.369 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:54:22.370 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:54:22.370 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:54:22.378 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: test 4 = 4 2026-02-18T08:54:22.379 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:54:22.379 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: jq .threshold 2026-02-18T08:54:22.389 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:54:22.389 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:54:22.390 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-18T08:54:22.390 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-18T08:54:22.390 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-18T08:54:22.391 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:54:22.391 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:22.391 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:22.391 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-mgr.x.asok 2026-02-18T08:54:22.391 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:54:22.391 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-mgr.x.asok dump_osd_network 0 2026-02-18T08:54:22.439 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:54:22.439 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 0, 2026-02-18T08:54:22.439 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [ 2026-02-18T08:54:22.439 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.439 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:17 2026", 2026-02-18T08:54:22.439 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.439 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:54:22.439 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.165, 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.191, 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.194 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.140, 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.140, 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.125 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.440 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.190, 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.326, 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.362 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.180 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:17 2026", 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:54:22.441 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.153, 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.170, 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.175 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.133, 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.123, 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.123 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.184, 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.306, 2026-02-18T08:54:22.442 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.310 2026-02-18T08:54:22.443 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.443 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.167 2026-02-18T08:54:22.443 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.443 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.443 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:16 2026", 2026-02-18T08:54:22.443 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.443 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:54:22.443 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:54:22.443 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:54:22.443 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.444 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.117, 2026-02-18T08:54:22.444 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.158, 2026-02-18T08:54:22.444 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.171 2026-02-18T08:54:22.444 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.444 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.444 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.090, 2026-02-18T08:54:22.444 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.090, 2026-02-18T08:54:22.444 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.090 2026-02-18T08:54:22.444 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.135, 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.208, 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.266 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.091 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:16 2026", 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:54:22.445 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.126, 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.165, 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.168 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.110, 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.110, 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.110 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.154, 2026-02-18T08:54:22.446 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.208, 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.255 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.127 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:16 2026", 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.134, 2026-02-18T08:54:22.447 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.159, 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.167 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.127, 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.122, 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.109 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.150, 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.220, 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.311 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.448 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.137 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:17 2026", 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.147, 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.167, 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.167 2026-02-18T08:54:22.449 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.132, 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.123, 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.114 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.164, 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.343, 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.343 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.145 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.450 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:16 2026", 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 2, 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.126, 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.147, 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.160 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.109, 2026-02-18T08:54:22.451 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.109, 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.109 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.141, 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.190, 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.297 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.116 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:16 2026", 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:54:22.452 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.125, 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.152, 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.157 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.116, 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.112, 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.106 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.453 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.140, 2026-02-18T08:54:22.454 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.208, 2026-02-18T08:54:22.454 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.288 2026-02-18T08:54:22.454 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.454 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.111 2026-02-18T08:54:22.454 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.454 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.454 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:17 2026", 2026-02-18T08:54:22.454 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.454 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 1, 2026-02-18T08:54:22.454 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.130, 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.148, 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.155 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.116, 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.116, 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.102 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.166, 2026-02-18T08:54:22.455 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.202, 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.303 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.123 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:18 2026", 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "front", 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.127, 2026-02-18T08:54:22.456 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.145, 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.155 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.115, 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.105, 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.097 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.144, 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.198, 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.248 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.457 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.127 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:18 2026", 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 0, 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 1, 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.104, 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.133, 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.148 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.458 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.095, 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.088, 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.088 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.111, 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.192, 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.233 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.105 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: { 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: "last update": "Wed Feb 18 08:54:16 2026", 2026-02-18T08:54:22.459 INFO:tasks.workunit.client.0.trial118.stdout: "stale": false, 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: "from osd": 2, 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: "to osd": 0, 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: "interface": "back", 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: "average": { 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.105, 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.127, 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.132 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: "min": { 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.098, 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.098, 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.091 2026-02-18T08:54:22.460 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.461 INFO:tasks.workunit.client.0.trial118.stdout: "max": { 2026-02-18T08:54:22.461 INFO:tasks.workunit.client.0.trial118.stdout: "1min": 0.119, 2026-02-18T08:54:22.461 INFO:tasks.workunit.client.0.trial118.stdout: "5min": 0.183, 2026-02-18T08:54:22.461 INFO:tasks.workunit.client.0.trial118.stdout: "15min": 0.205 2026-02-18T08:54:22.461 INFO:tasks.workunit.client.0.trial118.stdout: }, 2026-02-18T08:54:22.461 INFO:tasks.workunit.client.0.trial118.stdout: "last": 0.097 2026-02-18T08:54:22.461 INFO:tasks.workunit.client.0.trial118.stdout: } 2026-02-18T08:54:22.461 INFO:tasks.workunit.client.0.trial118.stdout: ] 2026-02-18T08:54:22.461 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:54:22.462 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:54:22.462 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: jq '.entries | length' 2026-02-18T08:54:22.462 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: test 12 = 12 2026-02-18T08:54:22.462 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:54:22.463 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: jq .threshold 2026-02-18T08:54:22.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: test 0 = 0 2026-02-18T08:54:22.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:54:22.466 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-18T08:54:22.466 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-18T08:54:22.467 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-18T08:54:22.467 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:54:22.467 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:22.467 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:22.467 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-mgr.x.asok 2026-02-18T08:54:22.468 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:54:22.468 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-mgr.x.asok dump_osd_network 99 2026-02-18T08:54:22.514 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:54:22.514 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 99, 2026-02-18T08:54:22.514 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [] 2026-02-18T08:54:22.515 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:54:22.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:54:22.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: jq .threshold 2026-02-18T08:54:22.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: test 99 = 99 2026-02-18T08:54:22.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: tee td/network-ping/json 2026-02-18T08:54:22.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: get_asok_path osd.0 2026-02-18T08:54:22.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:54:22.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:54:22.532 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:54:22.533 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:22.533 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:22.533 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-osd.0.asok 2026-02-18T08:54:22.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: CEPH_ARGS= 2026-02-18T08:54:22.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19969/ceph-osd.0.asok dump_osd_network 98 2026-02-18T08:54:22.585 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T08:54:22.585 INFO:tasks.workunit.client.0.trial118.stdout: "threshold": 98, 2026-02-18T08:54:22.585 INFO:tasks.workunit.client.0.trial118.stdout: "entries": [] 2026-02-18T08:54:22.585 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T08:54:22.592 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: cat td/network-ping/json 2026-02-18T08:54:22.592 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: jq .threshold 2026-02-18T08:54:22.602 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: test 98 = 98 2026-02-18T08:54:22.602 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:105: TEST_network_ping_test1: rm -f td/network-ping/json 2026-02-18T08:54:22.604 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-18T08:54:22.604 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-18T08:54:22.604 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:54:22.604 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-18T08:54:22.605 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:54:22.605 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:54:22.605 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:54:22.605 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:54:22.605 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:54:22.915 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:54:22.915 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:54:22.917 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:54:22.917 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:54:22.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:54:22.919 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:54:22.919 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:54:22.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:54:22.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:54:22.921 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:54:22.921 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:54:22.922 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:54:22.923 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:54:22.923 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-18T08:54:23.493 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:54:23.494 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:23.494 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:23.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19969 2026-02-18T08:54:23.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:54:23.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:54:23.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-18T08:54:23.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-18T08:54:23.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-18T08:54:23.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-18T08:54:23.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-18T08:54:23.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:54:23.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-18T08:54:23.496 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:54:23.496 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:54:23.497 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:54:23.497 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:54:23.497 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:54:23.498 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:54:23.498 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:54:23.499 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:54:23.499 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:54:23.500 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:54:23.500 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:54:23.500 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:54:23.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:54:23.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:54:23.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:54:23.502 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:54:23.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:54:23.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:54:23.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-18T08:54:23.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:54:23.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:23.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:23.506 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19969 2026-02-18T08:54:23.506 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:54:23.506 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:54:23.507 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-18T08:54:23.507 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:54:23.508 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:23.508 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:23.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19969 2026-02-18T08:54:23.509 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:54:23.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:54:23.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:54:23.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test2 td/network-ping 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:111: TEST_network_ping_test2: local dir=td/network-ping 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:113: TEST_network_ping_test2: export CEPH_ARGS 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: export 'EXTRA_OPTS= --mon_warn_on_slow_ping_time=0.001' 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: EXTRA_OPTS=' --mon_warn_on_slow_ping_time=0.001' 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:115: TEST_network_ping_test2: run_mon td/network-ping a 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-18T08:54:23.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-18T08:54:23.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:54:23.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:54:23.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:54:23.532 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:54:23.533 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:23.533 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:23.533 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:54:23.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --mon_warn_on_slow_ping_time=0.001 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:54:23.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:54:23.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:54:23.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:54:23.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:54:23.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:54:23.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:54:23.564 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:54:23.564 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:54:23.564 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:54:23.564 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:54:23.564 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:23.564 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:23.564 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-mon.a.asok 2026-02-18T08:54:23.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:54:23.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19969/ceph-mon.a.asok config get fsid 2026-02-18T08:54:23.618 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:54:23.618 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:54:23.619 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:54:23.619 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:54:23.619 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:54:23.620 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:54:23.620 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:54:23.620 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:54:23.620 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:54:23.620 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:23.620 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:23.620 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19969/ceph-mon.a.asok 2026-02-18T08:54:23.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:54:23.621 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19969/ceph-mon.a.asok config get mon_host 2026-02-18T08:54:23.674 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:116: TEST_network_ping_test2: run_mgr td/network-ping x 2026-02-18T08:54:23.674 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-18T08:54:23.674 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:54:23.675 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:54:23.675 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:54:23.675 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-18T08:54:23.675 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:54:23.785 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:54:23.786 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:54:23.786 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:54:23.786 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:54:23.786 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:23.786 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:23.786 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:54:23.787 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:54:23.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --mon_warn_on_slow_ping_time=0.001 --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:54:23.809 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:117: TEST_network_ping_test2: run_osd td/network-ping 0 2026-02-18T08:54:23.809 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-18T08:54:23.809 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:54:23.809 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:54:23.809 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:54:23.810 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:54:23.811 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:54:23.812 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:54:23.812 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:54:23.812 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:54:23.812 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:54:23.812 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-18T08:54:23.812 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:54:23.813 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4003102d-c9b4-4b91-9f87-195becfbd2a5 2026-02-18T08:54:23.813 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 4003102d-c9b4-4b91-9f87-195becfbd2a5 2026-02-18T08:54:23.813 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 4003102d-c9b4-4b91-9f87-195becfbd2a5' 2026-02-18T08:54:23.813 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:54:23.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA/fpVpZXiWMBAAJvhgeGY2hmmoGgG8uCi5wA== 2026-02-18T08:54:23.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA/fpVpZXiWMBAAJvhgeGY2hmmoGgG8uCi5wA=="}' 2026-02-18T08:54:23.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4003102d-c9b4-4b91-9f87-195becfbd2a5 -i td/network-ping/0/new.json 2026-02-18T08:54:23.938 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:54:23.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-18T08:54:23.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA/fpVpZXiWMBAAJvhgeGY2hmmoGgG8uCi5wA== --osd-uuid 4003102d-c9b4-4b91-9f87-195becfbd2a5 2026-02-18T08:54:23.965 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:23.951+0000 7f5e0dcdea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:23.966 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:23.955+0000 7f5e0dcdea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:23.967 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:23.955+0000 7f5e0dcdea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:23.968 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:23.955+0000 7f5e0dcdea40 -1 bdev(0x56195c98a400 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:54:23.968 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:23.955+0000 7f5e0dcdea40 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-18T08:54:24.654 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-18T08:54:24.654 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:54:24.654 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:54:24.655 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:54:24.655 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:54:24.779 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:54:24.779 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:54:24.780 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:54:24.780 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:54:24.780 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:54:24.780 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:54:24.804 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:24.795+0000 7f542e0b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:24.806 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:24.799+0000 7f542e0b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:24.808 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:24.799+0000 7f542e0b0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:24.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:54:24.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:54:24.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:54:24.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:54:24.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:54:24.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:54:24.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:54:24.888 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:54:24.888 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:54:24.888 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:54:24.993 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:54:25.196 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:25.187+0000 7f542e0b0a40 -1 Falling back to public interface 2026-02-18T08:54:25.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:54:25.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:54:25.994 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:54:25.995 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:54:25.995 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:54:25.995 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:54:26.100 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:54:26.127 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:26.118+0000 7f542e0b0a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:54:27.101 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:54:27.102 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:54:27.102 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:54:27.103 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:54:27.103 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:54:27.103 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:54:27.320 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2333721908,v1:127.0.0.1:6801/2333721908] [v2:127.0.0.1:6802/2333721908,v1:127.0.0.1:6803/2333721908] exists,up 4003102d-c9b4-4b91-9f87-195becfbd2a5 2026-02-18T08:54:27.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:54:27.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:54:27.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:54:27.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:118: TEST_network_ping_test2: run_osd td/network-ping 1 2026-02-18T08:54:27.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-18T08:54:27.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:54:27.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-18T08:54:27.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:54:27.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:54:27.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:54:27.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:54:27.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:54:27.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:54:27.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:54:27.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-18T08:54:27.327 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:54:27.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2b739a10-3e66-4932-9919-e4bde39b6b30 2026-02-18T08:54:27.328 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 2b739a10-3e66-4932-9919-e4bde39b6b30' 2026-02-18T08:54:27.328 INFO:tasks.workunit.client.0.trial118.stdout:add osd1 2b739a10-3e66-4932-9919-e4bde39b6b30 2026-02-18T08:54:27.328 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:54:27.337 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBDfpVpX5SdExAAoArYeDYRK0vDfe/lmqufJw== 2026-02-18T08:54:27.337 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBDfpVpX5SdExAAoArYeDYRK0vDfe/lmqufJw=="}' 2026-02-18T08:54:27.337 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2b739a10-3e66-4932-9919-e4bde39b6b30 -i td/network-ping/1/new.json 2026-02-18T08:54:27.556 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:54:27.566 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-18T08:54:27.567 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBDfpVpX5SdExAAoArYeDYRK0vDfe/lmqufJw== --osd-uuid 2b739a10-3e66-4932-9919-e4bde39b6b30 2026-02-18T08:54:27.584 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:27.574+0000 7f09369c3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:27.586 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:27.574+0000 7f09369c3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:27.586 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:27.578+0000 7f09369c3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:27.587 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:27.578+0000 7f09369c3a40 -1 bdev(0x563a5773e400 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-18T08:54:27.587 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:27.578+0000 7f09369c3a40 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-18T08:54:28.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-18T08:54:28.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:54:28.282 INFO:tasks.workunit.client.0.trial118.stdout:adding osd1 key to auth repository 2026-02-18T08:54:28.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-18T08:54:28.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:54:28.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-18T08:54:28.566 INFO:tasks.workunit.client.0.trial118.stdout:start osd.1 2026-02-18T08:54:28.567 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:54:28.567 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:54:28.567 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:54:28.567 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:54:28.591 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:28.582+0000 7fb7c0f40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:28.593 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:28.582+0000 7fb7c0f40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:28.594 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:28.586+0000 7fb7c0f40a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:28.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-18T08:54:28.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:54:28.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-18T08:54:28.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:54:28.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:54:28.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:54:28.786 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:54:28.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:54:28.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:54:28.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:54:28.972 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:28.962+0000 7fb7c0f40a40 -1 Falling back to public interface 2026-02-18T08:54:29.002 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:54:29.600 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:29.590+0000 7fb7c0f40a40 -1 osd.1 0 log_to_monitors true 2026-02-18T08:54:30.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:54:30.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:54:30.005 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:54:30.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:54:30.006 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:54:30.006 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:54:30.222 INFO:tasks.workunit.client.0.trial118.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3378641345,v1:127.0.0.1:6811/3378641345] [v2:127.0.0.1:6812/3378641345,v1:127.0.0.1:6813/3378641345] exists,up 2b739a10-3e66-4932-9919-e4bde39b6b30 2026-02-18T08:54:30.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:54:30.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:54:30.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:54:30.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:119: TEST_network_ping_test2: run_osd td/network-ping 2 2026-02-18T08:54:30.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-18T08:54:30.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:54:30.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-18T08:54:30.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:54:30.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-18T08:54:30.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-18T08:54:30.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:54:30.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:54:30.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:54:30.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-18T08:54:30.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-18T08:54:30.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:54:30.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-18T08:54:30.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-18T08:54:30.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-18T08:54:30.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:54:30.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:54:30.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:54:30.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:54:30.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:54:30.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:54:30.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-18T08:54:30.229 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:54:30.230 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cbb8a02f-a0b9-4937-8baa-2932c3562f54 2026-02-18T08:54:30.230 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 cbb8a02f-a0b9-4937-8baa-2932c3562f54' 2026-02-18T08:54:30.230 INFO:tasks.workunit.client.0.trial118.stdout:add osd2 cbb8a02f-a0b9-4937-8baa-2932c3562f54 2026-02-18T08:54:30.231 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:54:30.240 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBGfpVpEM3ODRAAq63/Q0fLpLCalnAwY9veEQ== 2026-02-18T08:54:30.241 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBGfpVpEM3ODRAAq63/Q0fLpLCalnAwY9veEQ=="}' 2026-02-18T08:54:30.241 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cbb8a02f-a0b9-4937-8baa-2932c3562f54 -i td/network-ping/2/new.json 2026-02-18T08:54:30.458 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:54:30.468 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-18T08:54:30.468 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBGfpVpEM3ODRAAq63/Q0fLpLCalnAwY9veEQ== --osd-uuid cbb8a02f-a0b9-4937-8baa-2932c3562f54 2026-02-18T08:54:30.485 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:30.474+0000 7f85f5772a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:30.486 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:30.478+0000 7f85f5772a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:30.487 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:30.478+0000 7f85f5772a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:30.488 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:30.478+0000 7f85f5772a40 -1 bdev(0x55ab671be400 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-18T08:54:30.488 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:30.478+0000 7f85f5772a40 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-18T08:54:31.173 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-18T08:54:31.173 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:54:31.174 INFO:tasks.workunit.client.0.trial118.stdout:adding osd2 key to auth repository 2026-02-18T08:54:31.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-18T08:54:31.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:54:31.465 INFO:tasks.workunit.client.0.trial118.stdout:start osd.2 2026-02-18T08:54:31.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-18T08:54:31.466 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=62be44ee-2e1a-4fc5-b919-906d92b378c2 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19969/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:54:31.466 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:54:31.466 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:54:31.466 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:54:31.489 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:31.478+0000 7f6abb7c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:31.491 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:31.482+0000 7f6abb7c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:31.492 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:31.482+0000 7f6abb7c9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:54:31.684 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-18T08:54:31.684 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:54:31.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-18T08:54:31.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:54:31.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:54:31.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:54:31.685 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:54:31.686 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:54:31.686 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:54:31.686 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-18T08:54:31.888 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:31.878+0000 7f6abb7c9a40 -1 Falling back to public interface 2026-02-18T08:54:31.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:54:32.473 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:54:32.462+0000 7f6abb7c9a40 -1 osd.2 0 log_to_monitors true 2026-02-18T08:54:32.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:54:32.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:54:32.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:54:32.896 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:54:32.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:54:32.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-18T08:54:33.111 INFO:tasks.workunit.client.0.trial118.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3988935472,v1:127.0.0.1:6819/3988935472] [v2:127.0.0.1:6820/3988935472,v1:127.0.0.1:6821/3988935472] exists,up cbb8a02f-a0b9-4937-8baa-2932c3562f54 2026-02-18T08:54:33.112 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:54:33.112 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:54:33.112 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:54:33.112 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:121: TEST_network_ping_test2: sleep 5 2026-02-18T08:54:38.112 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:122: TEST_network_ping_test2: ceph osd crush add-bucket dc1 datacenter 2026-02-18T08:54:38.377 INFO:tasks.workunit.client.0.trial118.stderr:bucket 'dc1' already exists 2026-02-18T08:54:38.388 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:123: TEST_network_ping_test2: ceph osd crush add-bucket dc2 datacenter 2026-02-18T08:54:38.645 INFO:tasks.workunit.client.0.trial118.stderr:bucket 'dc2' already exists 2026-02-18T08:54:38.654 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:124: TEST_network_ping_test2: ceph osd crush add-bucket dc3 datacenter 2026-02-18T08:54:38.924 INFO:tasks.workunit.client.0.trial118.stderr:bucket 'dc3' already exists 2026-02-18T08:54:38.933 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:125: TEST_network_ping_test2: ceph osd crush add-bucket rack1 rack 2026-02-18T08:54:39.205 INFO:tasks.workunit.client.0.trial118.stderr:bucket 'rack1' already exists 2026-02-18T08:54:39.215 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:126: TEST_network_ping_test2: ceph osd crush add-bucket rack2 rack 2026-02-18T08:54:39.470 INFO:tasks.workunit.client.0.trial118.stderr:bucket 'rack2' already exists 2026-02-18T08:54:39.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:127: TEST_network_ping_test2: ceph osd crush add-bucket rack3 rack 2026-02-18T08:54:39.732 INFO:tasks.workunit.client.0.trial118.stderr:bucket 'rack3' already exists 2026-02-18T08:54:39.741 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:128: TEST_network_ping_test2: ceph osd crush add-bucket host1 host 2026-02-18T08:54:40.011 INFO:tasks.workunit.client.0.trial118.stderr:bucket 'host1' already exists 2026-02-18T08:54:40.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:129: TEST_network_ping_test2: ceph osd crush add-bucket host2 host 2026-02-18T08:54:40.286 INFO:tasks.workunit.client.0.trial118.stderr:bucket 'host2' already exists 2026-02-18T08:54:40.295 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:130: TEST_network_ping_test2: ceph osd crush add-bucket host3 host 2026-02-18T08:54:40.560 INFO:tasks.workunit.client.0.trial118.stderr:bucket 'host3' already exists 2026-02-18T08:54:40.568 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:131: TEST_network_ping_test2: ceph osd crush move dc1 root=default 2026-02-18T08:54:40.843 INFO:tasks.workunit.client.0.trial118.stderr:no need to move item id -5 name 'dc1' to location {root=default} in crush map 2026-02-18T08:54:40.855 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:132: TEST_network_ping_test2: ceph osd crush move dc2 root=default 2026-02-18T08:54:41.111 INFO:tasks.workunit.client.0.trial118.stderr:no need to move item id -6 name 'dc2' to location {root=default} in crush map 2026-02-18T08:54:41.121 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:133: TEST_network_ping_test2: ceph osd crush move dc3 root=default 2026-02-18T08:54:41.391 INFO:tasks.workunit.client.0.trial118.stderr:no need to move item id -7 name 'dc3' to location {root=default} in crush map 2026-02-18T08:54:41.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:134: TEST_network_ping_test2: ceph osd crush move rack1 datacenter=dc1 2026-02-18T08:54:41.672 INFO:tasks.workunit.client.0.trial118.stderr:no need to move item id -8 name 'rack1' to location {datacenter=dc1} in crush map 2026-02-18T08:54:41.681 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:135: TEST_network_ping_test2: ceph osd crush move rack2 datacenter=dc2 2026-02-18T08:54:41.952 INFO:tasks.workunit.client.0.trial118.stderr:no need to move item id -9 name 'rack2' to location {datacenter=dc2} in crush map 2026-02-18T08:54:41.963 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:136: TEST_network_ping_test2: ceph osd crush move rack3 datacenter=dc3 2026-02-18T08:54:42.257 INFO:tasks.workunit.client.0.trial118.stderr:no need to move item id -10 name 'rack3' to location {datacenter=dc3} in crush map 2026-02-18T08:54:42.268 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:137: TEST_network_ping_test2: ceph osd crush move host1 rack=rack1 2026-02-18T08:54:42.528 INFO:tasks.workunit.client.0.trial118.stderr:no need to move item id -11 name 'host1' to location {rack=rack1} in crush map 2026-02-18T08:54:42.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:138: TEST_network_ping_test2: ceph osd crush move host2 rack=rack2 2026-02-18T08:54:42.806 INFO:tasks.workunit.client.0.trial118.stderr:no need to move item id -12 name 'host2' to location {rack=rack2} in crush map 2026-02-18T08:54:42.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:139: TEST_network_ping_test2: ceph osd crush move host3 rack=rack3 2026-02-18T08:54:43.085 INFO:tasks.workunit.client.0.trial118.stderr:no need to move item id -13 name 'host3' to location {rack=rack3} in crush map 2026-02-18T08:54:43.095 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:140: TEST_network_ping_test2: ceph osd crush set osd.0 1.0 host=host1 2026-02-18T08:54:43.361 INFO:tasks.workunit.client.0.trial118.stderr:set item id 0 name 'osd.0' weight 1 at location {host=host1}: no change 2026-02-18T08:54:43.370 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:141: TEST_network_ping_test2: ceph osd crush set osd.1 1.0 host=host2 2026-02-18T08:54:43.640 INFO:tasks.workunit.client.0.trial118.stderr:set item id 1 name 'osd.1' weight 1 at location {host=host2}: no change 2026-02-18T08:54:43.649 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:142: TEST_network_ping_test2: ceph osd crush set osd.2 1.0 host=host3 2026-02-18T08:54:43.912 INFO:tasks.workunit.client.0.trial118.stderr:set item id 2 name 'osd.2' weight 1 at location {host=host3}: no change 2026-02-18T08:54:43.921 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:143: TEST_network_ping_test2: ceph osd crush rule create-simple myrule default host firstn 2026-02-18T08:54:44.219 INFO:tasks.workunit.client.0.trial118.stderr:rule myrule already exists 2026-02-18T08:54:44.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:145: TEST_network_ping_test2: create_pool foo 16 16 replicated myrule 2026-02-18T08:54:44.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 16 replicated myrule 2026-02-18T08:54:44.502 INFO:tasks.workunit.client.0.trial118.stderr:pool 'foo' already exists 2026-02-18T08:54:44.512 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T08:54:45.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:148: TEST_network_ping_test2: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-18T08:54:45.534 INFO:tasks.workunit.client.0.trial118.stdout:hints = 1 2026-02-18T08:54:45.534 INFO:tasks.workunit.client.0.trial118.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-18T08:54:45.534 INFO:tasks.workunit.client.0.trial118.stdout:Object prefix: benchmark_data_trial118_31623 2026-02-18T08:54:45.534 INFO:tasks.workunit.client.0.trial118.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-18T08:54:45.534 INFO:tasks.workunit.client.0.trial118.stdout: 0 0 0 0 0 0 - 0 2026-02-18T08:54:46.534 INFO:tasks.workunit.client.0.trial118.stdout: 1 16 10724 10708 41.8245 41.8281 0.00117871 0.00149183 2026-02-18T08:54:47.534 INFO:tasks.workunit.client.0.trial118.stdout: 2 16 22072 22056 43.0745 44.3281 0.000789456 0.00144843 2026-02-18T08:54:48.535 INFO:tasks.workunit.client.0.trial118.stdout: 3 16 33457 33441 43.5394 44.4727 0.00126223 0.00143377 2026-02-18T08:54:49.535 INFO:tasks.workunit.client.0.trial118.stdout: 4 16 45040 45024 43.9652 45.2461 0.00154905 0.00141984 2026-02-18T08:54:50.535 INFO:tasks.workunit.client.0.trial118.stdout: 5 16 55990 55974 43.7265 42.7734 0.00166069 0.00142773 2026-02-18T08:54:51.535 INFO:tasks.workunit.client.0.trial118.stdout: 6 16 66967 66951 43.5847 42.8789 0.00104292 0.00143231 2026-02-18T08:54:52.535 INFO:tasks.workunit.client.0.trial118.stdout: 7 16 78385 78369 43.7295 44.6016 0.00120652 0.00142769 2026-02-18T08:54:53.535 INFO:tasks.workunit.client.0.trial118.stdout: 8 16 88636 88620 43.2683 40.043 0.00156359 0.00144294 2026-02-18T08:54:54.535 INFO:tasks.workunit.client.0.trial118.stdout: 9 16 98277 98261 42.6448 37.6602 0.00182036 0.00146405 2026-02-18T08:54:55.536 INFO:tasks.workunit.client.0.trial118.stdout: 10 5 108034 108029 42.1957 38.1562 0.00202287 0.00147967 2026-02-18T08:54:56.536 INFO:tasks.workunit.client.0.trial118.stdout:Total time run: 10.001 2026-02-18T08:54:56.536 INFO:tasks.workunit.client.0.trial118.stdout:Total writes made: 108034 2026-02-18T08:54:56.536 INFO:tasks.workunit.client.0.trial118.stdout:Write size: 4096 2026-02-18T08:54:56.536 INFO:tasks.workunit.client.0.trial118.stdout:Object size: 4096 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Bandwidth (MB/sec): 42.1966 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Stddev Bandwidth: 2.73421 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Max bandwidth (MB/sec): 45.2461 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Min bandwidth (MB/sec): 37.6602 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Average IOPS: 10802 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Stddev IOPS: 699.957 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Max IOPS: 11583 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Min IOPS: 9641 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Average Latency(s): 0.00147966 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Stddev Latency(s): 0.000683033 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Max latency(s): 0.0116651 2026-02-18T08:54:56.537 INFO:tasks.workunit.client.0.trial118.stdout:Min latency(s): 0.000424218 2026-02-18T08:54:56.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:151: TEST_network_ping_test2: sleep 10 2026-02-18T08:55:06.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:152: TEST_network_ping_test2: flush_pg_stats 2026-02-18T08:55:06.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:55:06.545 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:55:06.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-18T08:55:06.756 INFO:tasks.workunit.client.0.trial118.stderr:1 2026-02-18T08:55:06.757 INFO:tasks.workunit.client.0.trial118.stderr:2' 2026-02-18T08:55:06.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:55:06.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:06.757 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:55:06.837 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-18T08:55:06.838 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-18T08:55:06.838 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-18T08:55:06.838 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:06.838 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-18T08:55:06.915 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672970 2026-02-18T08:55:06.915 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672970 2026-02-18T08:55:06.915 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-42949672970' 2026-02-18T08:55:06.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:06.916 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-18T08:55:06.993 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-18T08:55:06.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-18T08:55:06.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490 1-42949672970 2-60129542153' 2026-02-18T08:55:06.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:06.994 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-18T08:55:06.994 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:06.995 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:55:06.995 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-18T08:55:06.996 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:06.997 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-18T08:55:06.997 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-18T08:55:06.997 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836490 2026-02-18T08:55:06.998 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:55:07.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-18T08:55:07.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:55:08.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:55:08.210 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:55:08.426 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836491 -lt 21474836490 2026-02-18T08:55:08.426 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:08.427 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672970 2026-02-18T08:55:08.427 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:08.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-18T08:55:08.428 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672970 2026-02-18T08:55:08.429 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:08.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672970 2026-02-18T08:55:08.430 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.1 seq 42949672970 2026-02-18T08:55:08.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-18T08:55:08.431 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T08:55:08.649 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-18T08:55:08.649 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:08.650 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542153 2026-02-18T08:55:08.650 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:08.651 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-18T08:55:08.652 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-18T08:55:08.652 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:08.653 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542153 2026-02-18T08:55:08.653 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-18T08:55:08.654 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.2 seq 60129542153 2026-02-18T08:55:08.654 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-18T08:55:08.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-18T08:55:08.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: ceph health 2026-02-18T08:55:08.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: tee td/network-ping/health 2026-02-18T08:55:09.114 INFO:tasks.workunit.client.0.trial118.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.430ms); Slow OSD heartbeats on front (longest 0.370ms) 2026-02-18T08:55:09.122 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:155: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats' td/network-ping/health 2026-02-18T08:55:09.124 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: ceph health detail 2026-02-18T08:55:09.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: tee td/network-ping/health 2026-02-18T08:55:09.406 INFO:tasks.workunit.client.0.trial118.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.430ms); Slow OSD heartbeats on front (longest 0.370ms) 2026-02-18T08:55:09.406 INFO:tasks.workunit.client.0.trial118.stdout:[WRN] OSD_SLOW_PING_TIME_BACK: Slow OSD heartbeats on back (longest 0.430ms) 2026-02-18T08:55:09.406 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.430 msec 2026-02-18T08:55:09.406 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.205 msec possibly improving 2026-02-18T08:55:09.406 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.174 msec possibly improving 2026-02-18T08:55:09.406 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.171 msec possibly improving 2026-02-18T08:55:09.407 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.169 msec possibly improving 2026-02-18T08:55:09.407 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.165 msec possibly improving 2026-02-18T08:55:09.407 INFO:tasks.workunit.client.0.trial118.stdout:[WRN] OSD_SLOW_PING_TIME_FRONT: Slow OSD heartbeats on front (longest 0.370ms) 2026-02-18T08:55:09.407 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.370 msec 2026-02-18T08:55:09.407 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.298 msec 2026-02-18T08:55:09.407 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.286 msec 2026-02-18T08:55:09.407 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.235 msec possibly improving 2026-02-18T08:55:09.407 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.211 msec 2026-02-18T08:55:09.407 INFO:tasks.workunit.client.0.trial118.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.200 msec 2026-02-18T08:55:09.415 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:158: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_BACK td/network-ping/health 2026-02-18T08:55:09.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:159: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_FRONT td/network-ping/health 2026-02-18T08:55:09.417 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:160: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats on front from osd[.][0-2] [[]dc[1-3],rack[1-3][]] to osd[.][0-2] [[]dc[1-3],rack[1-3][]]' td/network-ping/health 2026-02-18T08:55:09.419 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:162: TEST_network_ping_test2: rm -f td/network-ping/health 2026-02-18T08:55:09.420 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-18T08:55:09.420 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-18T08:55:09.421 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:55:09.421 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-18T08:55:09.421 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:55:09.422 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:55:09.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:55:09.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:55:09.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:55:09.732 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:55:09.733 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:55:09.734 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:55:09.734 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:55:09.736 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:55:09.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:55:09.737 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:55:09.738 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:55:09.738 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:55:09.738 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:55:09.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:55:09.739 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:55:09.740 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:55:09.740 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-18T08:55:10.425 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:55:10.425 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:10.425 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:55:10.425 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19969 2026-02-18T08:55:10.427 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:55:10.427 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:55:10.427 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-18T08:55:10.427 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/network-ping 0 2026-02-18T08:55:10.427 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-18T08:55:10.427 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-18T08:55:10.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-18T08:55:10.428 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:55:10.428 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:55:10.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:55:10.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:55:10.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:55:10.429 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:55:10.430 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:55:10.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:55:10.431 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:55:10.431 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:55:10.432 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:55:10.432 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:55:10.433 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:55:10.433 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:55:10.433 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:55:10.434 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:55:10.435 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:55:10.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-18T08:55:10.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-18T08:55:10.437 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:55:10.438 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:10.438 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19969 2026-02-18T08:55:10.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19969 2026-02-18T08:55:10.439 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:55:10.439 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:55:10.439 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-18T08:55:10.440 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-18T08:55:10.440 DEBUG:teuthology.orchestra.run.trial118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-18T08:55:10.492 INFO:tasks.workunit:Running workunit misc/ok-to-stop.sh... 2026-02-18T08:55:10.493 DEBUG:teuthology.orchestra.run.trial118:workunit test misc/ok-to-stop.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh 2026-02-18T08:55:10.546 INFO:tasks.workunit.client.0.trial118.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-18T08:55:10.550 INFO:tasks.workunit.client.0.trial118.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-18T08:55:10.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-18T08:55:10.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-18T08:55:10.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-18T08:55:10.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-18T08:55:10.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-18T08:55:10.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-18T08:55:10.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-18T08:55:10.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/ok-to-stop 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:6: run: local dir=td/ok-to-stop 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:7: run: shift 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: export CEPH_MON_A=127.0.0.1:7150 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: CEPH_MON_A=127.0.0.1:7150 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: export CEPH_MON_B=127.0.0.1:7151 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: CEPH_MON_B=127.0.0.1:7151 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: export CEPH_MON_C=127.0.0.1:7152 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: CEPH_MON_C=127.0.0.1:7152 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: export CEPH_MON_D=127.0.0.1:7153 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: CEPH_MON_D=127.0.0.1:7153 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: export CEPH_MON_E=127.0.0.1:7154 2026-02-18T08:55:10.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: CEPH_MON_E=127.0.0.1:7154 2026-02-18T08:55:10.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:14: run: export CEPH_ARGS 2026-02-18T08:55:10.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: uuidgen 2026-02-18T08:55:10.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: CEPH_ARGS+='--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none ' 2026-02-18T08:55:10.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: export 'ORIG_CEPH_ARGS=--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none ' 2026-02-18T08:55:10.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: ORIG_CEPH_ARGS='--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none ' 2026-02-18T08:55:10.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: set 2026-02-18T08:55:10.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-18T08:55:10.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: local 'funcs=TEST_0_mds 2026-02-18T08:55:10.555 INFO:tasks.workunit.client.0.trial118.stderr:TEST_0_osd 2026-02-18T08:55:10.555 INFO:tasks.workunit.client.0.trial118.stderr:TEST_1_mon_checks 2026-02-18T08:55:10.555 INFO:tasks.workunit.client.0.trial118.stderr:TEST_2_mons_checks 2026-02-18T08:55:10.555 INFO:tasks.workunit.client.0.trial118.stderr:TEST_3_mons_checks 2026-02-18T08:55:10.555 INFO:tasks.workunit.client.0.trial118.stderr:TEST_4_mons_checks 2026-02-18T08:55:10.555 INFO:tasks.workunit.client.0.trial118.stderr:TEST_5_mons_checks' 2026-02-18T08:55:10.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-18T08:55:10.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-18T08:55:10.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-18T08:55:10.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-18T08:55:10.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:55:10.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:55:10.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:55:10.556 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:55:10.556 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:55:10.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:55:10.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:55:10.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:55:10.557 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:55:10.557 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:55:10.557 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:55:10.558 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:55:10.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:55:10.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:55:10.559 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:55:10.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:55:10.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:55:10.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:55:10.561 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:55:10.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:55:10.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:55:10.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:55:10.565 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:55:10.565 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:10.565 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:10.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:55:10.566 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:55:10.566 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:55:10.566 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-18T08:55:10.567 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:55:10.567 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:10.567 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:10.567 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31971 2026-02-18T08:55:10.568 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:55:10.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-18T08:55:10.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-18T08:55:10.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:55:10.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-18T08:55:10.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_mds td/ok-to-stop 2026-02-18T08:55:10.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:208: TEST_0_mds: local dir=td/ok-to-stop 2026-02-18T08:55:10.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:210: TEST_0_mds: CEPH_ARGS='--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-18T08:55:10.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:212: TEST_0_mds: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-18T08:55:10.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:55:10.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:55:10.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:55:10.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:55:10.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-18T08:55:10.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-18T08:55:10.591 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:55:10.591 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:55:10.591 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:55:10.591 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:55:10.591 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:10.591 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:10.591 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:10.592 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-18T08:55:10.617 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:55:10.618 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:55:10.618 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:55:10.618 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:55:10.618 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:55:10.618 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:55:10.619 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:55:10.619 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:55:10.619 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:55:10.620 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:55:10.620 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:10.620 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:10.620 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:55:10.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:55:10.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get fsid 2026-02-18T08:55:10.690 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:55:10.690 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:55:10.690 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:55:10.690 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:55:10.690 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:55:10.690 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:55:10.690 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:55:10.690 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:55:10.691 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:55:10.691 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:10.691 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:10.691 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:55:10.691 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:55:10.692 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get mon_host 2026-02-18T08:55:10.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:213: TEST_0_mds: run_mgr td/ok-to-stop x 2026-02-18T08:55:10.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-18T08:55:10.748 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:55:10.748 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:55:10.748 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:55:10.748 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-18T08:55:10.748 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:55:10.862 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:55:10.862 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:55:10.862 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:55:10.862 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:55:10.862 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:10.863 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:10.863 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:10.863 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:55:10.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:55:10.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:214: TEST_0_mds: run_osd td/ok-to-stop 0 2026-02-18T08:55:10.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-18T08:55:10.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:55:10.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:55:10.885 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-18T08:55:10.886 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-18T08:55:10.886 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:55:10.886 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:55:10.886 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:55:10.886 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:55:10.886 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:55:10.886 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:55:10.886 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-18T08:55:10.886 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:55:10.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=723aea93-01a7-47c0-97c7-550314515f38 2026-02-18T08:55:10.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 723aea93-01a7-47c0-97c7-550314515f38' 2026-02-18T08:55:10.887 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 723aea93-01a7-47c0-97c7-550314515f38 2026-02-18T08:55:10.888 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:55:10.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBufpVp1DCuNBAA2WsvsKlLeMe4CWjoXnozQQ== 2026-02-18T08:55:10.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBufpVp1DCuNBAA2WsvsKlLeMe4CWjoXnozQQ=="}' 2026-02-18T08:55:10.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 723aea93-01a7-47c0-97c7-550314515f38 -i td/ok-to-stop/0/new.json 2026-02-18T08:55:11.004 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:55:11.012 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-18T08:55:11.014 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBufpVp1DCuNBAA2WsvsKlLeMe4CWjoXnozQQ== --osd-uuid 723aea93-01a7-47c0-97c7-550314515f38 2026-02-18T08:55:11.029 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:11.015+0000 7fa92311aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:11.030 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:11.015+0000 7fa92311aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:11.031 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:11.015+0000 7fa92311aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:11.032 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:11.015+0000 7fa92311aa40 -1 bdev(0x563cf959a400 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:55:11.032 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:11.015+0000 7fa92311aa40 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-18T08:55:11.844 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-18T08:55:11.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:55:11.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:55:11.845 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:55:11.846 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:55:11.967 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:55:11.968 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:55:11.968 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:55:11.968 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:55:11.968 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:55:11.968 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:55:11.991 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:11.975+0000 7f5c7942aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:11.994 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:11.979+0000 7f5c7942aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:11.995 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:11.979+0000 7f5c7942aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:12.077 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:55:12.077 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:55:12.078 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:55:12.078 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:55:12.078 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:55:12.078 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:12.078 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:55:12.078 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:55:12.078 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:12.078 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:55:12.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:55:12.530 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:12.515+0000 7f5c7942aa40 -1 Falling back to public interface 2026-02-18T08:55:13.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:55:13.189 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:13.189 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:55:13.190 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:55:13.190 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:13.190 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:55:13.394 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:13.379+0000 7f5c7942aa40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:55:13.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:55:14.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:55:14.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:14.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:55:14.400 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:55:14.401 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:14.401 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:55:14.616 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1405274612,v1:127.0.0.1:6801/1405274612] [v2:127.0.0.1:6802/1405274612,v1:127.0.0.1:6803/1405274612] exists,up 723aea93-01a7-47c0-97c7-550314515f38 2026-02-18T08:55:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:55:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:55:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:55:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:215: TEST_0_mds: run_mds td/ok-to-stop a 2026-02-18T08:55:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-18T08:55:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-18T08:55:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=a 2026-02-18T08:55:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-18T08:55:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/a 2026-02-18T08:55:14.619 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-18T08:55:14.619 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:55:14.619 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:55:14.619 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:55:14.619 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:14.619 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:14.619 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:14.619 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id a --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-18T08:55:14.640 INFO:tasks.workunit.client.0.trial118.stdout:starting mds.a at 2026-02-18T08:55:14.644 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:217: TEST_0_mds: ceph osd pool create meta 1 2026-02-18T08:55:14.919 INFO:tasks.workunit.client.0.trial118.stderr:pool 'meta' already exists 2026-02-18T08:55:14.928 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:218: TEST_0_mds: ceph osd pool create data 1 2026-02-18T08:55:15.224 INFO:tasks.workunit.client.0.trial118.stderr:pool 'data' already exists 2026-02-18T08:55:15.234 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:219: TEST_0_mds: ceph fs new myfs meta data 2026-02-18T08:55:15.578 INFO:tasks.workunit.client.0.trial118.stderr:filesystem 'myfs' already exists 2026-02-18T08:55:15.588 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:220: TEST_0_mds: sleep 5 2026-02-18T08:55:20.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:222: TEST_0_mds: ceph mds ok-to-stop a 2026-02-18T08:55:20.777 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 2026-02-18T08:55:20.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:223: TEST_0_mds: ceph mds ok-to-stop a dne 2026-02-18T08:55:20.972 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 2026-02-18T08:55:20.977 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:224: TEST_0_mds: ceph mds ok-to-stop dne 2026-02-18T08:55:21.257 INFO:tasks.workunit.client.0.trial118.stderr:should be safe to stop [dne] 2026-02-18T08:55:21.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:226: TEST_0_mds: run_mds td/ok-to-stop b 2026-02-18T08:55:21.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-18T08:55:21.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-18T08:55:21.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=b 2026-02-18T08:55:21.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-18T08:55:21.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/b 2026-02-18T08:55:21.270 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-18T08:55:21.270 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:55:21.270 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:55:21.271 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:55:21.271 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:21.271 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:21.271 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:21.271 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id b --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-18T08:55:21.294 INFO:tasks.workunit.client.0.trial118.stdout:starting mds.b at 2026-02-18T08:55:21.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:227: TEST_0_mds: sleep 5 2026-02-18T08:55:26.299 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:229: TEST_0_mds: ceph mds ok-to-stop a 2026-02-18T08:55:26.577 INFO:tasks.workunit.client.0.trial118.stderr:should be safe to stop [a] 2026-02-18T08:55:26.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:230: TEST_0_mds: ceph mds ok-to-stop b 2026-02-18T08:55:26.882 INFO:tasks.workunit.client.0.trial118.stderr:should be safe to stop [b] 2026-02-18T08:55:26.891 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:231: TEST_0_mds: ceph mds ok-to-stop a b 2026-02-18T08:55:27.095 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 4137 2026-02-18T08:55:27.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:232: TEST_0_mds: ceph mds ok-to-stop a dne1 dne2 2026-02-18T08:55:27.370 INFO:tasks.workunit.client.0.trial118.stderr:should be safe to stop [a,dne1,dne2] 2026-02-18T08:55:27.381 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:233: TEST_0_mds: ceph mds ok-to-stop b dne 2026-02-18T08:55:27.662 INFO:tasks.workunit.client.0.trial118.stderr:should be safe to stop [b,dne] 2026-02-18T08:55:27.671 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:234: TEST_0_mds: ceph mds ok-to-stop a b dne 2026-02-18T08:55:27.873 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 4137 2026-02-18T08:55:27.878 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:235: TEST_0_mds: ceph mds ok-to-stop dne1 dne2 2026-02-18T08:55:28.156 INFO:tasks.workunit.client.0.trial118.stderr:should be safe to stop [dne1,dne2] 2026-02-18T08:55:28.164 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:237: TEST_0_mds: kill_daemons td/ok-to-stop KILL mds.a 2026-02-18T08:55:28.165 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:55:28.165 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:55:28.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:55:28.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:55:28.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:55:28.273 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:55:28.273 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-18T08:55:28.274 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:55:28.274 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:55:28.274 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:55:28.274 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:55:28.274 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:55:28.383 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:55:28.384 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-18T08:55:28.384 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:55:28.384 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:55:28.384 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:55:28.384 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:55:28.384 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:55:28.384 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:55:28.384 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:55:28.384 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:55:28.389 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:55:28.389 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:55:28.390 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:55:28.391 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:55:28.392 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:55:28.392 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:55:28.393 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:55:28.394 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:55:28.394 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:55:28.394 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:55:28.394 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:55:28.395 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:55:28.397 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:55:28.397 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:55:28.403 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:55:28.403 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:28.403 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:28.403 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:55:28.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:55:28.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:55:28.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-18T08:55:28.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-18T08:55:28.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-18T08:55:28.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-18T08:55:28.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:55:28.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:55:28.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:55:28.406 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:55:28.406 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:55:28.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:55:28.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:55:28.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:55:28.407 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:55:28.408 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:55:28.409 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:55:28.409 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:55:28.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:55:28.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:55:28.410 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:55:28.411 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:55:28.411 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:55:28.412 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:55:28.412 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:55:28.413 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:55:28.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:55:28.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:55:28.415 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:55:28.416 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:28.416 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:28.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:55:28.417 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:55:28.417 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:55:28.417 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-18T08:55:28.418 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:55:28.418 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:28.419 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:28.419 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31971 2026-02-18T08:55:28.420 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:55:28.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:55:28.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:55:28.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-18T08:55:28.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_osd td/ok-to-stop 2026-02-18T08:55:28.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:241: TEST_0_osd: local dir=td/ok-to-stop 2026-02-18T08:55:28.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:243: TEST_0_osd: CEPH_ARGS='--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-18T08:55:28.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:245: TEST_0_osd: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-18T08:55:28.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:55:28.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:55:28.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:55:28.423 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:55:28.423 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-18T08:55:28.423 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-18T08:55:28.443 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:55:28.443 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:55:28.443 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:55:28.443 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:55:28.443 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:28.443 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:28.443 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:28.444 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-18T08:55:28.471 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:55:28.472 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:55:28.472 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:55:28.472 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:55:28.473 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:55:28.473 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:55:28.474 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:55:28.474 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:55:28.474 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:55:28.474 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:55:28.474 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:28.474 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:28.474 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:55:28.475 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:55:28.475 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get fsid 2026-02-18T08:55:28.530 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:55:28.530 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:55:28.530 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:55:28.530 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:55:28.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:55:28.531 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:55:28.531 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:55:28.531 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:55:28.532 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:55:28.532 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:28.532 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:28.532 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:55:28.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:55:28.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get mon_host 2026-02-18T08:55:28.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:246: TEST_0_osd: run_mgr td/ok-to-stop x 2026-02-18T08:55:28.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-18T08:55:28.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:55:28.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:55:28.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:55:28.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-18T08:55:28.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:55:28.695 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:55:28.695 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:55:28.695 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:55:28.695 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:55:28.695 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:28.696 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:28.696 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:28.696 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:55:28.697 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:55:28.718 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:247: TEST_0_osd: run_osd td/ok-to-stop 0 2026-02-18T08:55:28.718 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-18T08:55:28.718 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:55:28.718 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:55:28.718 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:55:28.718 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-18T08:55:28.718 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-18T08:55:28.718 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:55:28.718 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:28.719 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:55:28.720 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:55:28.721 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-18T08:55:28.721 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:55:28.721 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1f71ec50-e50d-479a-8477-1d70102097af 2026-02-18T08:55:28.722 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 1f71ec50-e50d-479a-8477-1d70102097af 2026-02-18T08:55:28.722 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1f71ec50-e50d-479a-8477-1d70102097af' 2026-02-18T08:55:28.722 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:55:28.730 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCAfpVpo37pKhAAM5OMZT1SXwu+Q0rk2p97xw== 2026-02-18T08:55:28.730 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCAfpVpo37pKhAAM5OMZT1SXwu+Q0rk2p97xw=="}' 2026-02-18T08:55:28.730 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1f71ec50-e50d-479a-8477-1d70102097af -i td/ok-to-stop/0/new.json 2026-02-18T08:55:28.840 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:55:28.848 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-18T08:55:28.849 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCAfpVpo37pKhAAM5OMZT1SXwu+Q0rk2p97xw== --osd-uuid 1f71ec50-e50d-479a-8477-1d70102097af 2026-02-18T08:55:28.863 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:28.847+0000 7fb4cd76ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:28.865 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:28.851+0000 7fb4cd76ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:28.865 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:28.851+0000 7fb4cd76ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:28.866 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:28.851+0000 7fb4cd76ca40 -1 bdev(0x5651b841c400 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:55:28.866 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:28.851+0000 7fb4cd76ca40 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-18T08:55:29.608 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-18T08:55:29.608 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:55:29.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:55:29.609 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:55:29.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:55:29.724 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:55:29.725 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:55:29.726 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:55:29.726 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:55:29.726 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:55:29.726 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:55:29.740 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:29.723+0000 7f68a1f29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:29.741 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:29.727+0000 7f68a1f29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:29.742 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:29.727+0000 7f68a1f29a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:29.833 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:55:29.834 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:55:29.834 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:55:29.834 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:55:29.834 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:55:29.834 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:29.834 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:55:29.834 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:55:29.835 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:29.835 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:55:29.942 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:55:30.214 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:30.199+0000 7f68a1f29a40 -1 Falling back to public interface 2026-02-18T08:55:30.934 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:30.919+0000 7f68a1f29a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:55:30.944 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:55:30.944 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:30.944 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:55:30.944 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:55:30.945 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:30.945 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:55:31.055 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:55:32.056 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:55:32.057 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:32.057 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:55:32.058 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:55:32.058 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:32.058 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:55:32.272 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1858380324,v1:127.0.0.1:6801/1858380324] [v2:127.0.0.1:6802/1858380324,v1:127.0.0.1:6803/1858380324] exists,up 1f71ec50-e50d-479a-8477-1d70102097af 2026-02-18T08:55:32.275 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:55:32.275 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:55:32.275 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:248: TEST_0_osd: run_osd td/ok-to-stop 1 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/1 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/1' 2026-02-18T08:55:32.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/1/journal' 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:32.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:55:32.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:55:32.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:55:32.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-18T08:55:32.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-18T08:55:32.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:55:32.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:55:32.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:55:32.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:55:32.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:55:32.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:55:32.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/1 2026-02-18T08:55:32.279 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:55:32.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=11287022-9e39-46e3-8f30-fa0b3b3ae686 2026-02-18T08:55:32.279 INFO:tasks.workunit.client.0.trial118.stdout:add osd1 11287022-9e39-46e3-8f30-fa0b3b3ae686 2026-02-18T08:55:32.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 11287022-9e39-46e3-8f30-fa0b3b3ae686' 2026-02-18T08:55:32.280 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:55:32.289 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCEfpVp6lmnEBAAWtTH01Q4RLKB/eymoxIv5w== 2026-02-18T08:55:32.289 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCEfpVp6lmnEBAAWtTH01Q4RLKB/eymoxIv5w=="}' 2026-02-18T08:55:32.289 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 11287022-9e39-46e3-8f30-fa0b3b3ae686 -i td/ok-to-stop/1/new.json 2026-02-18T08:55:32.513 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:55:32.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/1/new.json 2026-02-18T08:55:32.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCEfpVp6lmnEBAAWtTH01Q4RLKB/eymoxIv5w== --osd-uuid 11287022-9e39-46e3-8f30-fa0b3b3ae686 2026-02-18T08:55:32.548 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:32.535+0000 7fa3d228fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:32.550 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:32.535+0000 7fa3d228fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:32.551 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:32.535+0000 7fa3d228fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:32.551 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:32.535+0000 7fa3d228fa40 -1 bdev(0x56321d6be400 td/ok-to-stop/1/block) open stat got: (1) Operation not permitted 2026-02-18T08:55:32.552 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:32.535+0000 7fa3d228fa40 -1 bluestore(td/ok-to-stop/1) _read_fsid unparsable uuid 2026-02-18T08:55:33.243 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/1/keyring 2026-02-18T08:55:33.243 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:55:33.244 INFO:tasks.workunit.client.0.trial118.stdout:adding osd1 key to auth repository 2026-02-18T08:55:33.244 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-18T08:55:33.244 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:55:33.543 INFO:tasks.workunit.client.0.trial118.stdout:start osd.1 2026-02-18T08:55:33.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-18T08:55:33.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:55:33.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:55:33.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:55:33.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:55:33.565 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:33.551+0000 7fe69dd1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:33.566 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:33.551+0000 7fe69dd1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:33.567 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:33.551+0000 7fe69dd1fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:33.754 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-18T08:55:33.754 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:55:33.754 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-18T08:55:33.755 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:55:33.755 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:55:33.755 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:55:33.755 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:33.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:55:33.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:33.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:55:33.971 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:55:34.030 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:34.015+0000 7fe69dd1fa40 -1 Falling back to public interface 2026-02-18T08:55:34.621 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:34.607+0000 7fe69dd1fa40 -1 osd.1 0 log_to_monitors true 2026-02-18T08:55:34.972 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:55:34.972 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:34.972 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:55:34.972 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:55:34.972 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:34.972 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:55:35.181 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:55:36.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:55:36.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:36.183 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:55:36.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:55:36.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:36.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:55:36.398 INFO:tasks.workunit.client.0.trial118.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1386974692,v1:127.0.0.1:6811/1386974692] [v2:127.0.0.1:6812/1386974692,v1:127.0.0.1:6813/1386974692] exists,up 11287022-9e39-46e3-8f30-fa0b3b3ae686 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:249: TEST_0_osd: run_osd td/ok-to-stop 2 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/2 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/2' 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/2/journal' 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:36.399 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:55:36.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/2 2026-02-18T08:55:36.402 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:55:36.402 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fc7a3897-e9cc-4f59-afc3-4044f79810b1 2026-02-18T08:55:36.402 INFO:tasks.workunit.client.0.trial118.stdout:add osd2 fc7a3897-e9cc-4f59-afc3-4044f79810b1 2026-02-18T08:55:36.403 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 fc7a3897-e9cc-4f59-afc3-4044f79810b1' 2026-02-18T08:55:36.403 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:55:36.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCIfpVpdM8ZGBAAwcFEbiK9iorkpfdSkYsNXA== 2026-02-18T08:55:36.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCIfpVpdM8ZGBAAwcFEbiK9iorkpfdSkYsNXA=="}' 2026-02-18T08:55:36.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fc7a3897-e9cc-4f59-afc3-4044f79810b1 -i td/ok-to-stop/2/new.json 2026-02-18T08:55:36.631 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:55:36.640 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/2/new.json 2026-02-18T08:55:36.641 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCIfpVpdM8ZGBAAwcFEbiK9iorkpfdSkYsNXA== --osd-uuid fc7a3897-e9cc-4f59-afc3-4044f79810b1 2026-02-18T08:55:36.657 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:36.643+0000 7f7cbe194a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:36.659 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:36.643+0000 7f7cbe194a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:36.660 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:36.643+0000 7f7cbe194a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:36.660 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:36.647+0000 7f7cbe194a40 -1 bdev(0x5616d5eb8400 td/ok-to-stop/2/block) open stat got: (1) Operation not permitted 2026-02-18T08:55:36.660 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:36.647+0000 7f7cbe194a40 -1 bluestore(td/ok-to-stop/2) _read_fsid unparsable uuid 2026-02-18T08:55:37.318 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/2/keyring 2026-02-18T08:55:37.319 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:55:37.319 INFO:tasks.workunit.client.0.trial118.stdout:adding osd2 key to auth repository 2026-02-18T08:55:37.320 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-18T08:55:37.320 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:55:37.621 INFO:tasks.workunit.client.0.trial118.stdout:start osd.2 2026-02-18T08:55:37.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-18T08:55:37.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:55:37.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:55:37.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:55:37.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:55:37.642 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:37.627+0000 7f88ce079a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:37.644 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:37.627+0000 7f88ce079a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:37.645 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:37.631+0000 7f88ce079a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:37.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-18T08:55:37.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:55:37.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-18T08:55:37.828 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:55:37.828 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:55:37.828 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:37.828 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:55:37.828 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:55:37.829 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:37.829 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-18T08:55:38.049 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:55:38.050 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:38.035+0000 7f88ce079a40 -1 Falling back to public interface 2026-02-18T08:55:38.634 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:38.619+0000 7f88ce079a40 -1 osd.2 0 log_to_monitors true 2026-02-18T08:55:39.050 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:55:39.051 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:39.051 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:55:39.052 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:55:39.052 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:39.052 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-18T08:55:39.275 INFO:tasks.workunit.client.0.trial118.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/553805496,v1:127.0.0.1:6819/553805496] [v2:127.0.0.1:6820/553805496,v1:127.0.0.1:6821/553805496] exists,up fc7a3897-e9cc-4f59-afc3-4044f79810b1 2026-02-18T08:55:39.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:55:39.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:55:39.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:55:39.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:250: TEST_0_osd: run_osd td/ok-to-stop 3 2026-02-18T08:55:39.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-18T08:55:39.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:55:39.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-18T08:55:39.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:55:39.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/3 2026-02-18T08:55:39.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/3' 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/3/journal' 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:55:39.280 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:55:39.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:55:39.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:55:39.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:55:39.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:55:39.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/3 2026-02-18T08:55:39.282 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:55:39.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a9a71787-168a-4b74-8e8a-f79875c5fbee 2026-02-18T08:55:39.283 INFO:tasks.workunit.client.0.trial118.stdout:add osd3 a9a71787-168a-4b74-8e8a-f79875c5fbee 2026-02-18T08:55:39.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 a9a71787-168a-4b74-8e8a-f79875c5fbee' 2026-02-18T08:55:39.283 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:55:39.291 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCLfpVpCWW8EBAABik/CgBnea2bgNPIoHdXjA== 2026-02-18T08:55:39.291 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCLfpVpCWW8EBAABik/CgBnea2bgNPIoHdXjA=="}' 2026-02-18T08:55:39.291 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a9a71787-168a-4b74-8e8a-f79875c5fbee -i td/ok-to-stop/3/new.json 2026-02-18T08:55:39.513 INFO:tasks.workunit.client.0.trial118.stdout:3 2026-02-18T08:55:39.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/3/new.json 2026-02-18T08:55:39.523 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCLfpVpCWW8EBAABik/CgBnea2bgNPIoHdXjA== --osd-uuid a9a71787-168a-4b74-8e8a-f79875c5fbee 2026-02-18T08:55:39.539 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:39.523+0000 7f111620ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:39.541 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:39.527+0000 7f111620ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:39.542 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:39.527+0000 7f111620ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:39.542 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:39.527+0000 7f111620ca40 -1 bdev(0x5610cde56400 td/ok-to-stop/3/block) open stat got: (1) Operation not permitted 2026-02-18T08:55:39.542 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:39.527+0000 7f111620ca40 -1 bluestore(td/ok-to-stop/3) _read_fsid unparsable uuid 2026-02-18T08:55:40.159 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/3/keyring 2026-02-18T08:55:40.159 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:55:40.160 INFO:tasks.workunit.client.0.trial118.stdout:adding osd3 key to auth repository 2026-02-18T08:55:40.160 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-18T08:55:40.160 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:55:40.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-18T08:55:40.460 INFO:tasks.workunit.client.0.trial118.stdout:start osd.3 2026-02-18T08:55:40.460 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:55:40.461 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:55:40.461 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:55:40.461 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:55:40.482 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:40.467+0000 7f37f7929a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:40.484 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:40.467+0000 7f37f7929a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:40.485 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:40.471+0000 7f37f7929a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:55:40.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-18T08:55:40.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:55:40.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-18T08:55:40.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:55:40.681 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:55:40.681 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:40.681 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:55:40.682 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:55:40.682 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:40.682 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-18T08:55:40.894 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:40.879+0000 7f37f7929a40 -1 Falling back to public interface 2026-02-18T08:55:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:55:41.477 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:55:41.463+0000 7f37f7929a40 -1 osd.3 0 log_to_monitors true 2026-02-18T08:55:41.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:55:41.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:55:41.897 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:55:41.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:55:41.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:55:41.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-18T08:55:42.121 INFO:tasks.workunit.client.0.trial118.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3963286742,v1:127.0.0.1:6827/3963286742] [v2:127.0.0.1:6828/3963286742,v1:127.0.0.1:6829/3963286742] exists,up a9a71787-168a-4b74-8e8a-f79875c5fbee 2026-02-18T08:55:42.122 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:55:42.122 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:55:42.122 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:55:42.122 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:252: TEST_0_osd: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2026-02-18T08:55:42.357 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:253: TEST_0_osd: ceph osd pool create ec erasure ec-profile 2026-02-18T08:55:42.753 INFO:tasks.workunit.client.0.trial118.stderr:pool 'ec' already exists 2026-02-18T08:55:42.764 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:255: TEST_0_osd: wait_for_clean 2026-02-18T08:55:42.764 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T08:55:42.764 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T08:55:42.764 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T08:55:42.765 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T08:55:42.765 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T08:55:42.766 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T08:55:42.766 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T08:55:42.766 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T08:55:42.766 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T08:55:42.868 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T08:55:42.868 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T08:55:42.868 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T08:55:42.868 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T08:55:42.868 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:55:42.868 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:55:43.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-18T08:55:43.087 INFO:tasks.workunit.client.0.trial118.stderr:1 2026-02-18T08:55:43.087 INFO:tasks.workunit.client.0.trial118.stderr:2 2026-02-18T08:55:43.087 INFO:tasks.workunit.client.0.trial118.stderr:3' 2026-02-18T08:55:43.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:55:43.088 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:43.088 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:55:43.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-18T08:55:43.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-18T08:55:43.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485' 2026-02-18T08:55:43.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:43.166 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-18T08:55:43.244 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-18T08:55:43.244 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-18T08:55:43.244 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-18T08:55:43.244 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:43.244 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-18T08:55:43.319 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-18T08:55:43.319 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-18T08:55:43.320 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-60129542147' 2026-02-18T08:55:43.320 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:43.320 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-18T08:55:43.397 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-18T08:55:43.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-18T08:55:43.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-60129542147 3-77309411331' 2026-02-18T08:55:43.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:43.398 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836485 2026-02-18T08:55:43.399 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:43.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:55:43.399 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-18T08:55:43.400 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:43.401 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836485 2026-02-18T08:55:43.401 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836485 2026-02-18T08:55:43.402 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-18T08:55:43.402 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:55:43.612 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-18T08:55:43.613 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:55:44.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:55:44.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:55:44.830 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-18T08:55:44.830 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:55:45.831 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-18T08:55:45.832 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:55:46.044 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-18T08:55:46.045 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:46.045 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672964 2026-02-18T08:55:46.045 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:46.046 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-18T08:55:46.047 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-18T08:55:46.047 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:46.048 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672964 2026-02-18T08:55:46.049 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.1 seq 42949672964 2026-02-18T08:55:46.049 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-18T08:55:46.049 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T08:55:46.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-18T08:55:46.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:46.265 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542147 2026-02-18T08:55:46.265 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:46.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-18T08:55:46.266 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-18T08:55:46.266 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:46.268 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542147 2026-02-18T08:55:46.268 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.2 seq 60129542147 2026-02-18T08:55:46.268 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-18T08:55:46.269 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-18T08:55:46.486 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542148 -lt 60129542147 2026-02-18T08:55:46.487 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:46.487 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411331 2026-02-18T08:55:46.487 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:46.488 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-18T08:55:46.489 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411331 2026-02-18T08:55:46.489 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:46.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411331 2026-02-18T08:55:46.491 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.3 seq 77309411331 2026-02-18T08:55:46.491 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411331' 2026-02-18T08:55:46.491 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-18T08:55:46.713 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411331 -lt 77309411331 2026-02-18T08:55:46.714 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T08:55:46.714 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:55:46.714 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:55:46.999 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-18T08:55:46.999 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T08:55:46.999 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T08:55:46.999 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T08:55:47.000 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T08:55:47.000 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T08:55:47.000 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T08:55:47.000 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T08:55:47.213 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-18T08:55:47.214 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T08:55:47.214 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:55:47.214 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:55:47.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-18T08:55:47.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T08:55:47.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T08:55:47.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:258: TEST_0_osd: ceph osd pool set ec min_size 3 2026-02-18T08:55:47.756 INFO:tasks.workunit.client.0.trial118.stderr:set pool 1 min_size to 3 2026-02-18T08:55:47.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:259: TEST_0_osd: wait_for_clean 2026-02-18T08:55:47.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T08:55:47.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T08:55:47.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T08:55:47.767 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T08:55:47.767 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T08:55:47.767 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T08:55:47.768 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T08:55:47.768 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T08:55:47.768 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T08:55:47.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T08:55:47.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T08:55:47.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T08:55:47.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T08:55:47.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:55:47.879 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:55:48.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-18T08:55:48.099 INFO:tasks.workunit.client.0.trial118.stderr:1 2026-02-18T08:55:48.099 INFO:tasks.workunit.client.0.trial118.stderr:2 2026-02-18T08:55:48.100 INFO:tasks.workunit.client.0.trial118.stderr:3' 2026-02-18T08:55:48.100 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:55:48.100 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:48.100 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:55:48.176 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836488 2026-02-18T08:55:48.176 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836488 2026-02-18T08:55:48.176 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488' 2026-02-18T08:55:48.177 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:48.177 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-18T08:55:48.253 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672967 2026-02-18T08:55:48.253 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672967 2026-02-18T08:55:48.253 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672967' 2026-02-18T08:55:48.253 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:48.253 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-18T08:55:48.329 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542150 2026-02-18T08:55:48.329 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542150 2026-02-18T08:55:48.329 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672967 2-60129542150' 2026-02-18T08:55:48.329 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:48.329 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-18T08:55:48.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411334 2026-02-18T08:55:48.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411334 2026-02-18T08:55:48.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672967 2-60129542150 3-77309411334' 2026-02-18T08:55:48.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:48.406 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836488 2026-02-18T08:55:48.406 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:48.407 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:55:48.408 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836488 2026-02-18T08:55:48.408 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:48.409 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836488 2026-02-18T08:55:48.409 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836488 2026-02-18T08:55:48.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-18T08:55:48.410 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:55:48.629 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836485 -lt 21474836488 2026-02-18T08:55:48.629 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:55:49.630 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:55:49.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:55:49.847 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-18T08:55:49.848 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:49.848 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672967 2026-02-18T08:55:49.848 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:49.849 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-18T08:55:49.850 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672967 2026-02-18T08:55:49.850 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:49.851 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672967 2026-02-18T08:55:49.851 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.1 seq 42949672967 2026-02-18T08:55:49.852 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-18T08:55:49.852 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T08:55:50.071 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-18T08:55:50.071 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:50.072 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542150 2026-02-18T08:55:50.072 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:50.073 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-18T08:55:50.073 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542150 2026-02-18T08:55:50.074 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:50.075 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542150 2026-02-18T08:55:50.075 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-18T08:55:50.075 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.2 seq 60129542150 2026-02-18T08:55:50.075 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-18T08:55:50.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-18T08:55:50.284 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:50.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411334 2026-02-18T08:55:50.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:50.285 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-18T08:55:50.286 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411334 2026-02-18T08:55:50.286 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:50.287 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411334 2026-02-18T08:55:50.287 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.3 seq 77309411334 2026-02-18T08:55:50.288 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411334' 2026-02-18T08:55:50.288 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-18T08:55:50.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-18T08:55:50.495 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T08:55:50.495 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:55:50.496 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:55:50.775 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-18T08:55:50.776 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T08:55:50.776 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T08:55:50.776 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T08:55:50.776 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T08:55:50.776 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T08:55:50.776 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T08:55:50.776 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T08:55:50.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-18T08:55:50.988 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T08:55:50.988 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:55:50.988 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:55:51.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-18T08:55:51.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T08:55:51.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T08:55:51.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:261: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-18T08:55:51.464 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:51.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:262: TEST_0_osd: ceph osd ok-to-stop 1 2026-02-18T08:55:51.679 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:51.688 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:263: TEST_0_osd: ceph osd ok-to-stop 2 2026-02-18T08:55:51.908 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:51.917 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:264: TEST_0_osd: ceph osd ok-to-stop 3 2026-02-18T08:55:52.131 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:52.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:265: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-18T08:55:52.291 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":false,"osds":[0,1],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:52.292 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-18T08:55:52.296 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:266: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-18T08:55:52.449 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":false,"osds":[2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:52.449 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-18T08:55:52.454 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-18T08:55:52.454 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: grep '[0]' 2026-02-18T08:55:52.666 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:52.667 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: ceph osd ok-to-stop 1 --max 2 2026-02-18T08:55:52.667 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: grep '[1]' 2026-02-18T08:55:52.888 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:52.889 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:271: TEST_0_osd: ceph osd pool set ec min_size 2 2026-02-18T08:55:53.186 INFO:tasks.workunit.client.0.trial118.stderr:set pool 1 min_size to 2 2026-02-18T08:55:53.201 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:272: TEST_0_osd: wait_for_clean 2026-02-18T08:55:53.202 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T08:55:53.202 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T08:55:53.202 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T08:55:53.203 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T08:55:53.203 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T08:55:53.203 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T08:55:53.203 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T08:55:53.203 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T08:55:53.203 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T08:55:53.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T08:55:53.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T08:55:53.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T08:55:53.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T08:55:53.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:55:53.298 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:55:53.516 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-18T08:55:53.516 INFO:tasks.workunit.client.0.trial118.stderr:1 2026-02-18T08:55:53.516 INFO:tasks.workunit.client.0.trial118.stderr:2 2026-02-18T08:55:53.516 INFO:tasks.workunit.client.0.trial118.stderr:3' 2026-02-18T08:55:53.516 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:55:53.516 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:53.517 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:55:53.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-18T08:55:53.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-18T08:55:53.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836491' 2026-02-18T08:55:53.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:53.594 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-18T08:55:53.672 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672970 2026-02-18T08:55:53.672 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672970 2026-02-18T08:55:53.673 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836491 1-42949672970' 2026-02-18T08:55:53.673 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:53.673 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-18T08:55:53.750 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-18T08:55:53.750 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-18T08:55:53.750 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-60129542153' 2026-02-18T08:55:53.750 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:53.750 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-18T08:55:53.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411337 2026-02-18T08:55:53.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411337 2026-02-18T08:55:53.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-60129542153 3-77309411337' 2026-02-18T08:55:53.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:53.827 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836491 2026-02-18T08:55:53.828 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:53.828 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:55:53.829 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-18T08:55:53.829 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:53.830 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836491 2026-02-18T08:55:53.830 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-18T08:55:53.830 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836491 2026-02-18T08:55:53.831 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:55:54.045 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836489 -lt 21474836491 2026-02-18T08:55:54.045 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:55:55.046 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:55:55.047 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:55:55.253 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-18T08:55:55.253 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:55.254 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672970 2026-02-18T08:55:55.254 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:55.254 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-18T08:55:55.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672970 2026-02-18T08:55:55.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:55.256 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672970 2026-02-18T08:55:55.257 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.1 seq 42949672970 2026-02-18T08:55:55.257 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-18T08:55:55.257 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T08:55:55.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-18T08:55:55.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:55.475 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542153 2026-02-18T08:55:55.475 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:55.476 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-18T08:55:55.477 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-18T08:55:55.477 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:55.478 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542153 2026-02-18T08:55:55.478 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.2 seq 60129542153 2026-02-18T08:55:55.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-18T08:55:55.479 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-18T08:55:55.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542154 -lt 60129542153 2026-02-18T08:55:55.686 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:55.686 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411337 2026-02-18T08:55:55.686 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:55.687 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-18T08:55:55.688 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411337 2026-02-18T08:55:55.688 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:55.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411337 2026-02-18T08:55:55.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411337' 2026-02-18T08:55:55.690 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.3 seq 77309411337 2026-02-18T08:55:55.690 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-18T08:55:55.906 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411337 -lt 77309411337 2026-02-18T08:55:55.906 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T08:55:55.907 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:55:55.907 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:55:56.198 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-18T08:55:56.198 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T08:55:56.198 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T08:55:56.198 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T08:55:56.199 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T08:55:56.199 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T08:55:56.199 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T08:55:56.199 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T08:55:56.413 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-18T08:55:56.414 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T08:55:56.414 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:55:56.414 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:55:56.697 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-18T08:55:56.697 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T08:55:56.697 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T08:55:56.697 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:274: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-18T08:55:56.904 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:56.913 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:275: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-18T08:55:57.129 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[2,3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:57.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:276: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-18T08:55:57.305 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:57.306 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-18T08:55:57.309 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:277: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-18T08:55:57.466 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:57.466 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-18T08:55:57.470 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-18T08:55:57.471 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: grep '[0,1]' 2026-02-18T08:55:57.682 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:57.683 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: ceph osd ok-to-stop 0 --max 20 2026-02-18T08:55:57.683 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: grep '[0,1]' 2026-02-18T08:55:57.898 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:57.899 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: ceph osd ok-to-stop 2 --max 2 2026-02-18T08:55:57.899 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: grep '[2,3]' 2026-02-18T08:55:58.107 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:58.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: ceph osd ok-to-stop 2 --max 20 2026-02-18T08:55:58.108 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: grep '[2,3]' 2026-02-18T08:55:58.324 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:55:58.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:285: TEST_0_osd: kill_daemons td/ok-to-stop TERM osd.0 2026-02-18T08:55:58.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:55:58.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:55:58.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:55:58.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:55:58.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:55:58.633 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:55:58.633 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:286: TEST_0_osd: ceph osd down 0 2026-02-18T08:55:58.843 INFO:tasks.workunit.client.0.trial118.stderr:osd.0 is already down. 2026-02-18T08:55:58.853 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:287: TEST_0_osd: wait_for_peered 2026-02-18T08:55:58.854 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1731: wait_for_peered: local cmd= 2026-02-18T08:55:58.854 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1732: wait_for_peered: local num_peered=-1 2026-02-18T08:55:58.854 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1733: wait_for_peered: local cur_peered 2026-02-18T08:55:58.854 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: get_timeout_delays 90 .1 2026-02-18T08:55:58.855 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T08:55:58.855 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T08:55:58.855 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T08:55:58.855 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T08:55:58.855 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T08:55:58.961 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T08:55:58.961 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: local -a delays 2026-02-18T08:55:58.961 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1735: wait_for_peered: local -i loop=0 2026-02-18T08:55:58.961 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1737: wait_for_peered: flush_pg_stats 2026-02-18T08:55:58.961 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:55:58.961 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:55:59.171 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-18T08:55:59.171 INFO:tasks.workunit.client.0.trial118.stderr:1 2026-02-18T08:55:59.171 INFO:tasks.workunit.client.0.trial118.stderr:2 2026-02-18T08:55:59.171 INFO:tasks.workunit.client.0.trial118.stderr:3' 2026-02-18T08:55:59.171 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:55:59.171 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:59.171 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:55:59.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-18T08:55:59.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-18T08:55:59.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-18T08:55:59.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-18T08:55:59.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:59.235 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-18T08:55:59.314 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672973 2026-02-18T08:55:59.315 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672973 2026-02-18T08:55:59.315 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-42949672973' 2026-02-18T08:55:59.315 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:59.315 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-18T08:55:59.393 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542156 2026-02-18T08:55:59.393 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542156 2026-02-18T08:55:59.393 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-42949672973 2-60129542156' 2026-02-18T08:55:59.393 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:55:59.394 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-18T08:55:59.472 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411340 2026-02-18T08:55:59.472 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411340 2026-02-18T08:55:59.472 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-42949672973 2-60129542156 3-77309411340' 2026-02-18T08:55:59.472 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:55:59.473 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672973 2026-02-18T08:55:59.473 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:55:59.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-18T08:55:59.474 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672973 2026-02-18T08:55:59.475 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:55:59.476 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672973 2026-02-18T08:55:59.476 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.1 seq 42949672973 2026-02-18T08:55:59.477 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672973' 2026-02-18T08:55:59.477 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T08:55:59.695 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672971 -lt 42949672973 2026-02-18T08:55:59.696 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:56:00.696 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:56:00.697 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T08:56:00.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672971 -lt 42949672973 2026-02-18T08:56:00.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:56:01.913 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-18T08:56:01.914 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T08:56:02.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672974 -lt 42949672973 2026-02-18T08:56:02.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:56:02.133 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542156 2026-02-18T08:56:02.133 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:56:02.134 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-18T08:56:02.134 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542156 2026-02-18T08:56:02.134 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:56:02.136 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542156 2026-02-18T08:56:02.136 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.2 seq 60129542156 2026-02-18T08:56:02.136 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542156' 2026-02-18T08:56:02.137 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-18T08:56:02.365 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542157 -lt 60129542156 2026-02-18T08:56:02.365 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:56:02.365 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-77309411340 2026-02-18T08:56:02.366 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:56:02.367 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-18T08:56:02.367 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-77309411340 2026-02-18T08:56:02.367 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:56:02.369 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=77309411340 2026-02-18T08:56:02.369 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.3 seq 77309411340 2026-02-18T08:56:02.369 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 77309411340' 2026-02-18T08:56:02.370 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-18T08:56:02.583 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 77309411340 -lt 77309411340 2026-02-18T08:56:02.584 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: get_num_pgs 2026-02-18T08:56:02.585 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:56:02.585 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:56:02.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: test 32 == 0 2026-02-18T08:56:02.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1742: wait_for_peered: true 2026-02-18T08:56:02.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: get_num_active_or_peered 2026-02-18T08:56:02.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-18T08:56:02.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-18T08:56:02.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-18T08:56:02.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-18T08:56:02.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-18T08:56:03.079 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: cur_peered=32 2026-02-18T08:56:03.080 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: get_num_pgs 2026-02-18T08:56:03.080 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:56:03.080 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:56:03.372 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: test 32 = 32 2026-02-18T08:56:03.372 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: break 2026-02-18T08:56:03.372 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1762: wait_for_peered: return 0 2026-02-18T08:56:03.372 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:289: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-18T08:56:03.572 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":0,"num_not_ok_pgs":0} 2026-02-18T08:56:03.581 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:290: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-18T08:56:03.803 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:56:03.814 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:291: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-18T08:56:03.965 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:56:03.965 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-18T08:56:03.968 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:292: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-18T08:56:04.119 INFO:tasks.workunit.client.0.trial118.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-18T08:56:04.119 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-18T08:56:04.123 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:04.124 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:04.124 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:04.124 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:04.124 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:04.124 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:04.435 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:04.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-18T08:56:04.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:56:04.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:56:04.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:04.436 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:04.436 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:04.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:04.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:04.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:04.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:04.444 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:56:04.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:56:04.445 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:56:04.447 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:56:04.447 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:56:04.447 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:56:04.448 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:04.448 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:56:04.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:56:04.449 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:04.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:56:04.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:56:04.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:56:04.471 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:56:04.471 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:04.471 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:04.471 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:56:04.473 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:04.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:04.475 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:04.476 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:04.476 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:56:04.477 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:56:04.477 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:56:04.478 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:56:04.478 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:56:04.478 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:56:04.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:04.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:56:04.480 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:56:04.480 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:04.481 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:56:04.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:56:04.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:56:04.483 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:56:04.483 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:04.483 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:04.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:56:04.484 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:56:04.484 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:56:04.484 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-18T08:56:04.485 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:56:04.485 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:04.486 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:04.486 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31971 2026-02-18T08:56:04.487 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:56:04.488 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:56:04.488 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:56:04.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-18T08:56:04.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_1_mon_checks td/ok-to-stop 2026-02-18T08:56:04.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:28: TEST_1_mon_checks: local dir=td/ok-to-stop 2026-02-18T08:56:04.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:30: TEST_1_mon_checks: CEPH_ARGS='--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-18T08:56:04.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:32: TEST_1_mon_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-18T08:56:04.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:56:04.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:56:04.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:56:04.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:56:04.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-18T08:56:04.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-18T08:56:04.510 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:56:04.510 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:56:04.510 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:56:04.511 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:56:04.511 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:04.511 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:04.511 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:56:04.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-18T08:56:04.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:56:04.539 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:56:04.539 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:04.539 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:56:04.539 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:56:04.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:56:04.540 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:56:04.540 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:56:04.540 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:56:04.541 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:04.541 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:04.541 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:04.541 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:56:04.541 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:04.541 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get fsid 2026-02-18T08:56:04.597 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:56:04.597 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:04.597 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:56:04.597 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:56:04.598 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:56:04.598 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:56:04.598 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:56:04.598 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:56:04.598 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:04.598 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:04.599 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:04.599 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:56:04.599 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:04.599 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get mon_host 2026-02-18T08:56:04.654 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:34: TEST_1_mon_checks: ceph mon ok-to-stop dne 2026-02-18T08:56:04.757 INFO:tasks.workunit.client.0.trial118.stderr:invalid mon(s) specified: [dne] 2026-02-18T08:56:04.766 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:35: TEST_1_mon_checks: ceph mon ok-to-stop a 2026-02-18T08:56:04.852 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a] 2026-02-18T08:56:04.855 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:37: TEST_1_mon_checks: ceph mon ok-to-add-offline 2026-02-18T08:56:04.939 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-18T08:56:04.942 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:39: TEST_1_mon_checks: ceph mon ok-to-rm a 2026-02-18T08:56:05.027 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-18T08:56:05.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:40: TEST_1_mon_checks: ceph mon ok-to-rm dne 2026-02-18T08:56:05.146 INFO:tasks.workunit.client.0.trial118.stderr:mon.dne does not exist 2026-02-18T08:56:05.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:05.157 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:05.157 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:05.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:05.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:05.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:05.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:05.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-18T08:56:05.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:56:05.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:56:05.265 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:05.265 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:05.266 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:05.266 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:05.266 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:05.266 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:05.271 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:05.271 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:56:05.272 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:56:05.272 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:56:05.273 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:56:05.273 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:56:05.274 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:56:05.275 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:05.275 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:56:05.275 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:56:05.276 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:05.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:56:05.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:56:05.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:56:05.280 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:56:05.280 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:05.280 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:05.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:56:05.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:56:05.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:56:05.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-18T08:56:05.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-18T08:56:05.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-18T08:56:05.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-18T08:56:05.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:56:05.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:56:05.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:05.283 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:05.283 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:05.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:05.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:05.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:05.284 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:05.285 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:56:05.285 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:56:05.286 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:56:05.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:56:05.287 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:56:05.287 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:56:05.288 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:05.288 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:56:05.288 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:56:05.289 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:05.290 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:56:05.291 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:56:05.291 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:56:05.292 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:56:05.292 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:05.292 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:05.292 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:56:05.293 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:56:05.293 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:56:05.293 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-18T08:56:05.295 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:56:05.295 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:05.295 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:05.295 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31971 2026-02-18T08:56:05.296 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:56:05.297 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:56:05.297 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:56:05.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-18T08:56:05.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_2_mons_checks td/ok-to-stop 2026-02-18T08:56:05.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:44: TEST_2_mons_checks: local dir=td/ok-to-stop 2026-02-18T08:56:05.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:46: TEST_2_mons_checks: CEPH_ARGS='--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151 ' 2026-02-18T08:56:05.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:48: TEST_2_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-18T08:56:05.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:56:05.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:56:05.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:56:05.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:56:05.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-18T08:56:05.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-18T08:56:05.317 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:56:05.317 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:56:05.317 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:56:05.318 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:56:05.318 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:05.318 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:05.318 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:56:05.318 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-18T08:56:05.346 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:56:05.348 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:56:05.348 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:05.348 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:56:05.348 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:56:05.348 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:56:05.349 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:56:05.349 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:56:05.349 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:56:05.349 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:05.350 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:05.350 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:05.350 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:56:05.350 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:05.350 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get fsid 2026-02-18T08:56:05.404 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:56:05.404 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:05.404 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:56:05.404 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:56:05.405 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:56:05.405 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:56:05.405 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:56:05.405 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:56:05.406 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:05.406 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:05.406 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:05.406 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:56:05.406 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:05.406 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get mon_host 2026-02-18T08:56:05.460 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:49: TEST_2_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-18T08:56:05.460 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:56:05.460 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:56:05.460 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-18T08:56:05.460 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:56:05.460 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-18T08:56:05.460 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-18T08:56:05.478 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:56:05.478 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:56:05.478 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:56:05.479 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:56:05.479 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:05.479 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:05.479 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:56:05.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-18T08:56:05.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:56:05.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-18T08:56:05.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:05.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-18T08:56:05.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:56:05.506 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:56:05.506 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-18T08:56:05.506 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-18T08:56:05.507 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-18T08:56:05.507 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:05.507 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:05.507 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:05.507 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.b.asok 2026-02-18T08:56:05.507 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:05.508 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.b.asok config get fsid 2026-02-18T08:56:05.572 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-18T08:56:05.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:05.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-18T08:56:05.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:56:05.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:56:05.574 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-18T08:56:05.574 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-18T08:56:05.574 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-18T08:56:05.574 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:05.574 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:05.575 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:05.575 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.b.asok 2026-02-18T08:56:05.575 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:05.575 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.b.asok config get mon_host 2026-02-18T08:56:05.628 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:51: TEST_2_mons_checks: ceph mon ok-to-stop dne 2026-02-18T08:56:05.731 INFO:tasks.workunit.client.0.trial118.stderr:invalid mon(s) specified: [dne] 2026-02-18T08:56:05.738 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:52: TEST_2_mons_checks: ceph mon ok-to-stop a 2026-02-18T08:56:05.836 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (b) after stopping mons [a] 2026-02-18T08:56:05.840 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:53: TEST_2_mons_checks: ceph mon ok-to-stop b 2026-02-18T08:56:05.926 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b] 2026-02-18T08:56:05.929 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:54: TEST_2_mons_checks: ceph mon ok-to-stop a b 2026-02-18T08:56:06.014 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b] 2026-02-18T08:56:06.017 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:56: TEST_2_mons_checks: ceph mon ok-to-add-offline 2026-02-18T08:56:06.123 INFO:tasks.workunit.client.0.trial118.stderr:adding another mon that is not yet online will not break quorum 2026-02-18T08:56:06.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:58: TEST_2_mons_checks: ceph mon ok-to-rm a 2026-02-18T08:56:06.237 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.a 2026-02-18T08:56:06.245 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:59: TEST_2_mons_checks: ceph mon ok-to-rm b 2026-02-18T08:56:06.363 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.b 2026-02-18T08:56:06.374 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:60: TEST_2_mons_checks: ceph mon ok-to-rm dne 2026-02-18T08:56:06.479 INFO:tasks.workunit.client.0.trial118.stderr:mon.dne does not exist 2026-02-18T08:56:06.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:06.491 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:06.491 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:06.491 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:06.491 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:06.491 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:06.598 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:06.598 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-18T08:56:06.598 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:56:06.599 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:56:06.599 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:06.599 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:06.599 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:06.599 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:06.599 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:06.599 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:06.604 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:06.605 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:56:06.606 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:56:06.606 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:56:06.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:56:06.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:56:06.607 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:56:06.608 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:06.608 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:56:06.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:56:06.609 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:06.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:56:06.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:56:06.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:56:06.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:56:06.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:06.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:06.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:56:06.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:56:06.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:56:06.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-18T08:56:06.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-18T08:56:06.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-18T08:56:06.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-18T08:56:06.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:56:06.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:56:06.616 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:06.616 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:06.616 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:06.616 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:06.616 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:06.616 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:06.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:06.619 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:56:06.619 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:56:06.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:56:06.620 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:56:06.620 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:56:06.621 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:56:06.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:06.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:56:06.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:56:06.623 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:06.624 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:56:06.624 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:56:06.625 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:56:06.626 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:56:06.626 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:06.626 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:06.626 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:56:06.627 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:56:06.627 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:56:06.627 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-18T08:56:06.628 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:56:06.628 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:06.628 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:06.629 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31971 2026-02-18T08:56:06.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_3_mons_checks td/ok-to-stop 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:64: TEST_3_mons_checks: local dir=td/ok-to-stop 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:66: TEST_3_mons_checks: CEPH_ARGS='--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152 ' 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:68: TEST_3_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-18T08:56:06.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-18T08:56:06.651 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:56:06.651 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:56:06.651 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:56:06.652 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:56:06.652 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:06.652 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:06.652 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:56:06.652 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-18T08:56:06.682 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:56:06.683 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:56:06.683 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:06.683 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:56:06.683 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:56:06.683 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:56:06.683 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:56:06.683 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:56:06.683 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:56:06.684 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:06.684 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:06.684 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:06.684 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:56:06.684 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:06.684 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get fsid 2026-02-18T08:56:06.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:56:06.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:06.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:56:06.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:56:06.739 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:56:06.739 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:56:06.739 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:56:06.739 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:56:06.740 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:06.740 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:06.740 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:06.740 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:56:06.740 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:06.740 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get mon_host 2026-02-18T08:56:06.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:69: TEST_3_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-18T08:56:06.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:56:06.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:56:06.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-18T08:56:06.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:56:06.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-18T08:56:06.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-18T08:56:06.813 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:56:06.813 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:56:06.813 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:56:06.813 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:56:06.813 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:06.813 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:06.813 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:56:06.814 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-18T08:56:06.838 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:56:06.839 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-18T08:56:06.839 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:06.839 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-18T08:56:06.839 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:56:06.839 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:56:06.840 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-18T08:56:06.840 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-18T08:56:06.840 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-18T08:56:06.840 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:06.840 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:06.840 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:06.840 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.b.asok 2026-02-18T08:56:06.841 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:06.841 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.b.asok config get fsid 2026-02-18T08:56:06.907 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-18T08:56:06.907 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:06.907 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-18T08:56:06.907 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:56:06.907 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:56:06.908 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-18T08:56:06.908 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-18T08:56:06.908 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-18T08:56:06.908 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:06.908 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:06.908 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:06.909 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.b.asok 2026-02-18T08:56:06.909 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:06.909 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.b.asok config get mon_host 2026-02-18T08:56:06.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:70: TEST_3_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-18T08:56:06.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:56:06.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:56:06.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-18T08:56:06.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:56:06.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-18T08:56:06.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-18T08:56:06.984 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:56:06.985 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:56:06.985 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:56:06.986 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:56:06.986 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:06.986 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:06.986 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:56:06.986 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-18T08:56:07.021 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:56:07.022 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-18T08:56:07.022 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:07.022 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-18T08:56:07.022 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:56:07.022 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:56:07.023 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-18T08:56:07.023 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-18T08:56:07.024 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-18T08:56:07.024 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:07.024 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:07.024 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:07.024 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.c.asok 2026-02-18T08:56:07.024 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:07.024 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.c.asok config get fsid 2026-02-18T08:56:07.078 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-18T08:56:07.078 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:07.078 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-18T08:56:07.078 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:56:07.078 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:56:07.079 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-18T08:56:07.080 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-18T08:56:07.080 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-18T08:56:07.080 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:07.080 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:07.080 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:07.080 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.c.asok 2026-02-18T08:56:07.080 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:07.080 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.c.asok config get mon_host 2026-02-18T08:56:07.134 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:71: TEST_3_mons_checks: wait_for_quorum 60 3 2026-02-18T08:56:07.134 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-18T08:56:07.134 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-18T08:56:07.134 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-18T08:56:07.134 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-18T08:56:07.135 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-18T08:56:07.135 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-18T08:56:07.135 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771405027 2026-02-18T08:56:07.135 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:07.136 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404967 -lt 1771405027 ]] 2026-02-18T08:56:07.137 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:07.137 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:07.454 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:07.455 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:07.455 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:07.456 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:07.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-18T08:56:07.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-18T08:56:07.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-18T08:56:07.466 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-18T08:56:07.466 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-18T08:56:07.466 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:73: TEST_3_mons_checks: ceph mon ok-to-stop dne 2026-02-18T08:56:07.569 INFO:tasks.workunit.client.0.trial118.stderr:invalid mon(s) specified: [dne] 2026-02-18T08:56:07.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:74: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-18T08:56:07.682 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (b,c) after stopping [a] 2026-02-18T08:56:07.692 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:75: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-18T08:56:07.798 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-18T08:56:07.807 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:76: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-18T08:56:07.914 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (a,b) after stopping [c] 2026-02-18T08:56:07.924 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:77: TEST_3_mons_checks: ceph mon ok-to-stop a b 2026-02-18T08:56:08.013 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a,b] 2026-02-18T08:56:08.017 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:78: TEST_3_mons_checks: ceph mon ok-to-stop b c 2026-02-18T08:56:08.104 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b,c] 2026-02-18T08:56:08.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:79: TEST_3_mons_checks: ceph mon ok-to-stop a b c 2026-02-18T08:56:08.194 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b,c] 2026-02-18T08:56:08.197 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:81: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-18T08:56:08.304 INFO:tasks.workunit.client.0.trial118.stderr:adding another mon that is not yet online will not break quorum 2026-02-18T08:56:08.312 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:83: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-18T08:56:08.428 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.a 2026-02-18T08:56:08.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:84: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-18T08:56:08.544 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.b 2026-02-18T08:56:08.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:85: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-18T08:56:08.670 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.c 2026-02-18T08:56:08.678 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:87: TEST_3_mons_checks: kill_daemons td/ok-to-stop KILL mon.b 2026-02-18T08:56:08.679 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:08.679 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:08.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:08.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:08.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:08.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:08.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:88: TEST_3_mons_checks: wait_for_quorum 60 2 2026-02-18T08:56:08.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-18T08:56:08.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=2 2026-02-18T08:56:08.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-18T08:56:08.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 2 ]] 2026-02-18T08:56:08.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-18T08:56:08.787 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-18T08:56:08.788 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771405028 2026-02-18T08:56:08.788 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:08.789 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404968 -lt 1771405028 ]] 2026-02-18T08:56:08.789 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:08.789 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:08.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:08.898 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:08.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:08.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:08.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:08.909 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:08.909 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:08.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404968 -lt 1771405028 ]] 2026-02-18T08:56:08.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:08.910 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:09.024 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:09.024 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.025 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:09.025 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:09.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:09.036 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:09.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404969 -lt 1771405028 ]] 2026-02-18T08:56:09.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:09.037 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:09.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:09.147 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.147 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:09.147 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:09.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:09.157 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:09.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404969 -lt 1771405028 ]] 2026-02-18T08:56:09.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:09.159 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:09.271 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:09.271 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.272 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:09.272 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:09.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:09.281 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:09.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404969 -lt 1771405028 ]] 2026-02-18T08:56:09.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:09.282 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:09.392 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:09.393 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.393 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.393 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:09.404 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:09.404 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:09.404 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:09.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404969 -lt 1771405028 ]] 2026-02-18T08:56:09.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:09.406 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:09.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:09.520 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.520 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:09.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:09.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:09.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:09.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404969 -lt 1771405028 ]] 2026-02-18T08:56:09.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:09.533 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:09.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:09.646 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.647 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:09.647 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.656 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:09.656 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:09.656 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:09.657 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404969 -lt 1771405028 ]] 2026-02-18T08:56:09.657 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:09.658 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:09.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:09.773 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.773 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:09.774 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:09.783 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:09.783 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:09.784 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:09.784 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404969 -lt 1771405028 ]] 2026-02-18T08:56:09.785 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:09.785 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:12.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:12.895 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:12.896 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:12.896 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:12.905 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:12.905 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:12.906 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:12.907 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404972 -lt 1771405028 ]] 2026-02-18T08:56:12.907 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:12.907 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:13.018 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:13.019 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:13.019 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:13.019 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:13.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:13.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:13.029 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:13.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404973 -lt 1771405028 ]] 2026-02-18T08:56:13.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:13.030 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:16.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:16.143 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.143 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:16.144 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:16.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:16.153 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:16.154 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404976 -lt 1771405028 ]] 2026-02-18T08:56:16.154 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:16.155 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:16.268 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:16.268 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.269 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.269 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:16.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:16.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:16.280 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:16.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404976 -lt 1771405028 ]] 2026-02-18T08:56:16.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:16.282 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:16.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:16.396 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.397 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.397 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:16.407 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:16.408 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:16.408 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:16.409 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404976 -lt 1771405028 ]] 2026-02-18T08:56:16.409 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:16.409 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:16.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:16.523 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.523 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.523 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:16.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:16.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:16.534 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:16.535 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404976 -lt 1771405028 ]] 2026-02-18T08:56:16.535 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:16.535 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:16.648 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:16.648 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.649 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.649 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:16.659 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:16.659 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:16.660 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:16.661 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404976 -lt 1771405028 ]] 2026-02-18T08:56:16.661 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:16.661 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:16.772 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:16.772 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.773 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.773 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:16.784 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:16.784 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:16.785 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:16.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404976 -lt 1771405028 ]] 2026-02-18T08:56:16.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:16.786 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:16.901 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:16.901 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.902 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:16.902 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:16.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:16.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:16.912 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:16.913 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404976 -lt 1771405028 ]] 2026-02-18T08:56:16.913 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:16.913 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:17.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:17.029 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:17.029 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:17.029 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:17.040 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:17.040 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:17.040 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:17.041 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404977 -lt 1771405028 ]] 2026-02-18T08:56:17.041 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-18T08:56:17.041 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:23.766 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:23.767 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:23.767 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-18T08:56:23.768 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:06.635224Z","created":"2026-02-18T08:56:06.635224Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635221+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635222+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:06:635223+0000","crush_location":[]}]}}' 2026-02-18T08:56:23.776 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-18T08:56:23.776 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-18T08:56:23.777 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-18T08:56:23.777 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-18T08:56:23.777 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-18T08:56:23.777 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:90: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-18T08:56:23.863 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a] 2026-02-18T08:56:23.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:91: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-18T08:56:23.972 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-18T08:56:23.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:92: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-18T08:56:24.080 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [c] 2026-02-18T08:56:24.082 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:94: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-18T08:56:24.169 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-18T08:56:24.172 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:96: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-18T08:56:24.258 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-18T08:56:24.261 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:97: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-18T08:56:24.369 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.b 2026-02-18T08:56:24.377 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:98: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-18T08:56:24.477 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-18T08:56:24.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:24.479 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:24.479 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:24.480 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:24.480 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:24.480 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:24.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:24.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-18T08:56:24.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:56:24.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:56:24.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:24.589 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:24.589 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:24.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:24.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:24.590 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:24.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:24.595 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:56:24.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:56:24.596 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:56:24.597 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:56:24.597 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:56:24.598 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:56:24.599 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:24.599 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:56:24.599 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:56:24.600 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:24.600 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:56:24.601 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:56:24.601 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:56:24.607 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:56:24.607 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:24.607 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:24.608 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:56:24.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:56:24.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:56:24.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-18T08:56:24.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-18T08:56:24.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-18T08:56:24.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-18T08:56:24.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:56:24.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:56:24.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:56:24.610 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:24.610 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:24.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:24.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:24.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:24.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:24.612 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:56:24.613 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:56:24.613 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:56:24.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:56:24.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:56:24.615 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:56:24.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:24.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:56:24.616 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:56:24.616 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:56:24.617 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:56:24.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:56:24.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:56:24.619 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:56:24.619 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:24.619 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:24.619 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:56:24.620 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:56:24.620 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:56:24.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-18T08:56:24.622 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:56:24.622 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:24.622 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:24.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31971 2026-02-18T08:56:24.623 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:56:24.625 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:56:24.625 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:56:24.625 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-18T08:56:24.625 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_4_mons_checks td/ok-to-stop 2026-02-18T08:56:24.625 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:102: TEST_4_mons_checks: local dir=td/ok-to-stop 2026-02-18T08:56:24.625 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:104: TEST_4_mons_checks: CEPH_ARGS='--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153 ' 2026-02-18T08:56:24.625 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:106: TEST_4_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-18T08:56:24.625 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:56:24.626 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:56:24.626 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:56:24.626 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:56:24.626 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-18T08:56:24.626 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-18T08:56:24.646 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:56:24.646 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:56:24.646 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:56:24.646 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:56:24.646 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:24.646 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:24.647 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:56:24.647 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-18T08:56:24.673 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:56:24.674 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:56:24.674 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:24.674 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:56:24.674 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:56:24.676 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:56:24.676 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:56:24.676 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:56:24.676 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:56:24.676 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:24.676 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:24.676 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:24.676 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:56:24.676 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:24.676 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get fsid 2026-02-18T08:56:24.730 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:56:24.730 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:24.730 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:56:24.731 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:56:24.732 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:56:24.732 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:56:24.732 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:56:24.732 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:56:24.732 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:24.732 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:24.732 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:24.732 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:56:24.733 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:24.733 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get mon_host 2026-02-18T08:56:24.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:107: TEST_4_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-18T08:56:24.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:56:24.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:56:24.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-18T08:56:24.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:56:24.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-18T08:56:24.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-18T08:56:24.805 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:56:24.805 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:56:24.805 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:56:24.806 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:56:24.806 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:24.806 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:24.806 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:56:24.806 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-18T08:56:24.829 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:56:24.830 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-18T08:56:24.830 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:24.830 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-18T08:56:24.830 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:56:24.830 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:56:24.831 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-18T08:56:24.831 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-18T08:56:24.831 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-18T08:56:24.832 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:24.832 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:24.832 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:24.832 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.b.asok 2026-02-18T08:56:24.832 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:24.832 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.b.asok config get fsid 2026-02-18T08:56:24.885 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-18T08:56:24.885 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:24.886 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-18T08:56:24.886 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:56:24.887 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:56:24.887 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-18T08:56:24.887 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-18T08:56:24.887 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-18T08:56:24.887 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:24.887 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:24.887 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:24.888 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.b.asok 2026-02-18T08:56:24.888 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:24.888 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.b.asok config get mon_host 2026-02-18T08:56:24.941 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:108: TEST_4_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-18T08:56:24.941 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:56:24.941 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:56:24.941 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-18T08:56:24.941 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:56:24.941 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-18T08:56:24.941 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-18T08:56:24.960 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:56:24.960 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:56:24.961 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:56:24.961 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:56:24.961 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:24.961 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:24.961 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:56:24.961 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-18T08:56:24.995 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:56:24.996 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-18T08:56:24.996 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:24.996 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-18T08:56:24.996 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:56:24.997 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:56:24.997 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-18T08:56:24.997 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-18T08:56:24.997 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-18T08:56:24.997 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:24.997 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:24.998 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:24.998 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.c.asok 2026-02-18T08:56:24.998 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:24.998 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.c.asok config get fsid 2026-02-18T08:56:25.059 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-18T08:56:25.060 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:25.060 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-18T08:56:25.060 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:56:25.060 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:56:25.061 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-18T08:56:25.061 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-18T08:56:25.061 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-18T08:56:25.061 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:25.061 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:25.061 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:25.061 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.c.asok 2026-02-18T08:56:25.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:25.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.c.asok config get mon_host 2026-02-18T08:56:25.114 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:109: TEST_4_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-18T08:56:25.115 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:56:25.115 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:56:25.115 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-18T08:56:25.115 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:56:25.115 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-18T08:56:25.115 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-18T08:56:25.133 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:56:25.134 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:56:25.134 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:56:25.134 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:56:25.134 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:25.134 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:25.134 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:56:25.135 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-18T08:56:25.163 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:56:25.164 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-18T08:56:25.164 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:25.164 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-18T08:56:25.164 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:56:25.164 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:56:25.165 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-18T08:56:25.166 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-18T08:56:25.166 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-18T08:56:25.166 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:25.166 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:25.166 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:25.166 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.d.asok 2026-02-18T08:56:25.166 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:25.166 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.d.asok config get fsid 2026-02-18T08:56:25.221 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-18T08:56:25.221 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:56:25.221 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-18T08:56:25.221 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:56:25.221 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:56:25.221 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-18T08:56:25.221 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-18T08:56:25.222 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-18T08:56:25.222 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:56:25.223 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:56:25.223 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:56:25.223 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.d.asok 2026-02-18T08:56:25.223 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:56:25.223 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.d.asok config get mon_host 2026-02-18T08:56:25.275 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:110: TEST_4_mons_checks: wait_for_quorum 60 4 2026-02-18T08:56:25.275 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-18T08:56:25.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-18T08:56:25.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-18T08:56:25.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-18T08:56:25.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-18T08:56:25.276 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-18T08:56:25.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771405045 2026-02-18T08:56:25.277 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:25.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771404985 -lt 1771405045 ]] 2026-02-18T08:56:25.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:56:25.278 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:31.397 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:31.397 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:31.398 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:56:31.398 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:31.407 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-18T08:56:31.408 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-18T08:56:31.408 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-18T08:56:31.408 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-18T08:56:31.408 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-18T08:56:31.408 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:112: TEST_4_mons_checks: ceph mon ok-to-stop dne 2026-02-18T08:56:31.511 INFO:tasks.workunit.client.0.trial118.stderr:invalid mon(s) specified: [dne] 2026-02-18T08:56:31.521 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:113: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-18T08:56:31.623 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-18T08:56:31.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:114: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-18T08:56:31.735 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (a,c,d) after stopping [b] 2026-02-18T08:56:31.743 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:115: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-18T08:56:34.851 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (a,b,d) after stopping [c] 2026-02-18T08:56:34.859 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:116: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-18T08:56:37.965 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (a,b,c) after stopping [d] 2026-02-18T08:56:37.975 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:117: TEST_4_mons_checks: ceph mon ok-to-stop a b 2026-02-18T08:56:38.061 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [a,b] 2026-02-18T08:56:38.065 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:118: TEST_4_mons_checks: ceph mon ok-to-stop c d 2026-02-18T08:56:38.152 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (a,b) after stopping mons [c,d] 2026-02-18T08:56:38.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:120: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-18T08:56:38.261 INFO:tasks.workunit.client.0.trial118.stderr:adding another mon that is not yet online will not break quorum 2026-02-18T08:56:38.271 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:122: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-18T08:56:41.377 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.a 2026-02-18T08:56:41.385 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:123: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-18T08:56:41.498 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.b 2026-02-18T08:56:41.506 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:124: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-18T08:56:41.620 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.c 2026-02-18T08:56:41.628 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:126: TEST_4_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-18T08:56:41.629 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:56:41.629 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:56:41.629 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:56:41.629 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:56:41.629 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:56:41.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:56:41.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:127: TEST_4_mons_checks: wait_for_quorum 60 3 2026-02-18T08:56:41.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-18T08:56:41.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-18T08:56:41.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-18T08:56:41.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-18T08:56:41.738 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-18T08:56:41.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-18T08:56:41.738 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771405061 2026-02-18T08:56:41.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:41.739 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405001 -lt 1771405061 ]] 2026-02-18T08:56:41.739 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:41.740 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:41.849 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:41.849 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:41.849 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:41.850 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:41.860 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:41.860 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:41.860 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:41.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405001 -lt 1771405061 ]] 2026-02-18T08:56:41.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:41.861 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:44.976 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:44.976 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:44.977 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:44.977 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:44.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:44.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:44.988 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:44.988 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405004 -lt 1771405061 ]] 2026-02-18T08:56:44.989 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:44.989 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:45.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:45.104 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:45.105 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:45.105 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:45.115 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:45.115 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:45.116 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:45.117 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405005 -lt 1771405061 ]] 2026-02-18T08:56:45.117 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:45.117 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:45.233 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:45.233 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:45.234 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:45.234 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:45.243 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:45.243 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:45.244 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:45.245 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405005 -lt 1771405061 ]] 2026-02-18T08:56:45.245 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:45.245 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:45.354 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:45.355 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:45.355 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:45.355 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:45.364 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:45.364 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:45.365 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:45.366 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405005 -lt 1771405061 ]] 2026-02-18T08:56:45.366 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:45.366 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:45.478 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:45.478 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:45.479 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:45.479 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:45.488 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:45.488 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:45.489 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:45.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405005 -lt 1771405061 ]] 2026-02-18T08:56:45.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:45.490 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:48.601 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:48.602 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:48.602 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:48.602 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:48.612 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:48.612 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:48.612 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:48.613 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405008 -lt 1771405061 ]] 2026-02-18T08:56:48.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:48.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:48.726 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:48.726 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:48.727 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:48.727 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:48.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:48.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:48.737 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:48.738 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405008 -lt 1771405061 ]] 2026-02-18T08:56:48.738 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:48.739 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:48.853 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:48.853 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:48.854 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:48.854 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:48.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:48.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:48.864 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:48.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405008 -lt 1771405061 ]] 2026-02-18T08:56:48.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:48.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:48.978 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:48.978 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:48.979 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:48.979 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:48.988 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:48.988 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:48.989 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:48.990 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405008 -lt 1771405061 ]] 2026-02-18T08:56:48.990 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:48.990 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:49.102 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:49.103 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:49.103 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:49.103 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:49.112 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:56:49.113 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:56:49.113 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:56:49.114 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405009 -lt 1771405061 ]] 2026-02-18T08:56:49.114 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:56:49.114 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:56:58.231 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:56:58.231 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":10,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:58.232 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:56:58.232 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:56:24.629403Z","created":"2026-02-18T08:56:24.629403Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629399+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629401+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:56:24:629402+0000","crush_location":[]}]}}' 2026-02-18T08:56:58.241 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-18T08:56:58.241 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-18T08:56:58.241 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-18T08:56:58.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-18T08:56:58.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-18T08:56:58.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:129: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-18T08:56:58.344 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-18T08:56:58.354 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:130: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-18T08:57:01.443 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-18T08:57:01.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:131: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-18T08:57:01.530 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-18T08:57:01.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:132: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-18T08:57:01.616 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-18T08:57:01.620 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:134: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-18T08:57:01.725 INFO:tasks.workunit.client.0.trial118.stderr:adding another mon that is not yet online will not break quorum 2026-02-18T08:57:01.735 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:136: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-18T08:57:04.839 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.a 2026-02-18T08:57:04.847 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:137: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-18T08:57:07.961 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.b 2026-02-18T08:57:07.969 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:138: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-18T08:57:08.081 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.c 2026-02-18T08:57:08.088 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:139: TEST_4_mons_checks: ceph mon ok-to-rm d 2026-02-18T08:57:08.205 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.d 2026-02-18T08:57:08.213 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-18T08:57:08.214 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:57:08.214 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:57:08.214 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:57:08.214 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:57:08.214 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:57:08.323 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:57:08.323 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-18T08:57:08.323 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:57:08.323 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:57:08.323 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:57:08.323 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:57:08.323 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:57:08.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:57:08.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:57:08.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:57:08.328 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:57:08.329 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:57:08.329 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:57:08.330 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:57:08.331 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:57:08.331 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:57:08.331 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:57:08.332 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:57:08.333 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:57:08.333 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:57:08.333 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:57:08.334 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:57:08.335 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:57:08.335 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:57:08.345 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:57:08.345 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.345 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.345 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:57:08.347 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:57:08.347 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:57:08.348 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-18T08:57:08.348 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-18T08:57:08.348 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-18T08:57:08.348 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-18T08:57:08.348 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:57:08.348 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:57:08.348 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:57:08.348 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:57:08.348 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:57:08.348 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:57:08.348 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:57:08.349 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:57:08.349 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:57:08.350 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:57:08.351 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:57:08.351 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:57:08.353 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:57:08.353 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:57:08.353 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:57:08.354 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:57:08.354 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:57:08.355 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:57:08.355 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:57:08.356 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:57:08.357 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:57:08.357 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:57:08.358 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:57:08.358 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.359 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.359 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:57:08.360 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:57:08.360 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:57:08.360 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-18T08:57:08.361 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:57:08.361 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.361 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.361 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31971 2026-02-18T08:57:08.362 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:57:08.364 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:57:08.364 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:57:08.364 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-18T08:57:08.364 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_5_mons_checks td/ok-to-stop 2026-02-18T08:57:08.364 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:143: TEST_5_mons_checks: local dir=td/ok-to-stop 2026-02-18T08:57:08.364 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:145: TEST_5_mons_checks: CEPH_ARGS='--fsid=45b8fc6e-2391-45d7-88d7-7bf91afa6519 --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153,127.0.0.1:7154 ' 2026-02-18T08:57:08.364 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:147: TEST_5_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-18T08:57:08.364 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:57:08.365 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:57:08.365 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:57:08.365 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:57:08.365 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-18T08:57:08.365 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-18T08:57:08.383 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:57:08.384 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:57:08.384 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:57:08.384 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:57:08.384 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.384 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.384 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:57:08.385 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-18T08:57:08.412 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:57:08.413 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:57:08.413 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:57:08.413 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:57:08.413 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:57:08.415 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:57:08.415 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:57:08.415 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:57:08.415 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:57:08.415 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:57:08.415 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.415 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.415 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:57:08.416 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:57:08.416 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get fsid 2026-02-18T08:57:08.469 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:57:08.469 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:57:08.470 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:57:08.470 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:57:08.470 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:57:08.471 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:57:08.471 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:57:08.471 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:57:08.471 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:57:08.471 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.471 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.471 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.a.asok 2026-02-18T08:57:08.472 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:57:08.472 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.a.asok config get mon_host 2026-02-18T08:57:08.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:148: TEST_5_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-18T08:57:08.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:57:08.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:57:08.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-18T08:57:08.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:57:08.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-18T08:57:08.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-18T08:57:08.542 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:57:08.542 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:57:08.543 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:57:08.543 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:57:08.544 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.544 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.544 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:57:08.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-18T08:57:08.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:57:08.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-18T08:57:08.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:57:08.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-18T08:57:08.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:57:08.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:57:08.574 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-18T08:57:08.574 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-18T08:57:08.575 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-18T08:57:08.575 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:57:08.575 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.575 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.575 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.b.asok 2026-02-18T08:57:08.575 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:57:08.575 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.b.asok config get fsid 2026-02-18T08:57:08.629 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-18T08:57:08.629 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:57:08.629 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-18T08:57:08.629 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:57:08.629 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:57:08.631 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-18T08:57:08.631 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-18T08:57:08.631 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-18T08:57:08.631 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:57:08.631 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.631 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.631 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.b.asok 2026-02-18T08:57:08.631 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:57:08.631 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.b.asok config get mon_host 2026-02-18T08:57:08.684 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:149: TEST_5_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-18T08:57:08.684 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:57:08.684 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:57:08.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-18T08:57:08.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:57:08.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-18T08:57:08.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-18T08:57:08.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:57:08.705 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:57:08.705 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:57:08.705 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:57:08.705 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.705 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.706 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:57:08.706 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-18T08:57:08.735 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:57:08.736 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-18T08:57:08.736 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:57:08.736 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-18T08:57:08.736 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:57:08.736 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:57:08.737 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-18T08:57:08.737 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-18T08:57:08.738 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-18T08:57:08.738 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:57:08.738 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.738 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.738 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.c.asok 2026-02-18T08:57:08.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:57:08.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.c.asok config get fsid 2026-02-18T08:57:08.792 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-18T08:57:08.792 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:57:08.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-18T08:57:08.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:57:08.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:57:08.794 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-18T08:57:08.794 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-18T08:57:08.794 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-18T08:57:08.794 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:57:08.794 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.794 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.794 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.c.asok 2026-02-18T08:57:08.795 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:57:08.795 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.c.asok config get mon_host 2026-02-18T08:57:08.850 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:150: TEST_5_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-18T08:57:08.850 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:57:08.850 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:57:08.850 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-18T08:57:08.850 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:57:08.850 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-18T08:57:08.850 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-18T08:57:08.870 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:57:08.870 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:57:08.870 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:57:08.870 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:57:08.870 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.870 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.871 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:57:08.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-18T08:57:08.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:57:08.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-18T08:57:08.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:57:08.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-18T08:57:08.900 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:57:08.900 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:57:08.901 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-18T08:57:08.901 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-18T08:57:08.901 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-18T08:57:08.901 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:57:08.901 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.902 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.902 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.d.asok 2026-02-18T08:57:08.902 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:57:08.902 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.d.asok config get fsid 2026-02-18T08:57:08.957 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-18T08:57:08.958 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:57:08.958 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-18T08:57:08.958 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:57:08.958 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:57:08.958 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-18T08:57:08.958 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-18T08:57:08.958 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-18T08:57:08.958 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:57:08.958 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:08.958 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:08.958 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.d.asok 2026-02-18T08:57:08.959 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:57:08.959 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.d.asok config get mon_host 2026-02-18T08:57:09.012 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:151: TEST_5_mons_checks: run_mon td/ok-to-stop e --public-addr=127.0.0.1:7154 2026-02-18T08:57:09.012 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-18T08:57:09.013 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:57:09.013 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=e 2026-02-18T08:57:09.013 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:57:09.013 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/e 2026-02-18T08:57:09.013 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id e --mkfs --mon-data=td/ok-to-stop/e --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7154 2026-02-18T08:57:09.032 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:57:09.032 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:57:09.032 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:57:09.033 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:57:09.033 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:09.034 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:09.034 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31971/$cluster-$name.asok' 2026-02-18T08:57:09.034 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id e --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/e '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31971/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7154 2026-02-18T08:57:09.060 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:57:09.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e fsid 2026-02-18T08:57:09.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:57:09.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-18T08:57:09.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:57:09.063 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:57:09.063 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-18T08:57:09.064 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-18T08:57:09.064 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-18T08:57:09.064 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:57:09.064 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:09.064 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:09.064 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.e.asok 2026-02-18T08:57:09.064 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:57:09.064 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.e.asok config get fsid 2026-02-18T08:57:09.119 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e mon_host 2026-02-18T08:57:09.119 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:57:09.119 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-18T08:57:09.119 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:57:09.119 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:57:09.119 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-18T08:57:09.119 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-18T08:57:09.119 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-18T08:57:09.120 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:57:09.120 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:57:09.121 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:57:09.121 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31971/ceph-mon.e.asok 2026-02-18T08:57:09.121 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:57:09.121 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31971/ceph-mon.e.asok config get mon_host 2026-02-18T08:57:09.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:152: TEST_5_mons_checks: wait_for_quorum 60 5 2026-02-18T08:57:09.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-18T08:57:09.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=5 2026-02-18T08:57:09.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-18T08:57:09.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 5 ]] 2026-02-18T08:57:09.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-18T08:57:09.175 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-18T08:57:09.176 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771405089 2026-02-18T08:57:09.176 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:09.177 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405029 -lt 1771405089 ]] 2026-02-18T08:57:09.177 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 5' 2026-02-18T08:57:09.178 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:15.292 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:15.292 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:15.293 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 5' 2026-02-18T08:57:15.293 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:15.303 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:15.303 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:15.304 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:15.305 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405035 -lt 1771405089 ]] 2026-02-18T08:57:15.305 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 5' 2026-02-18T08:57:15.305 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:15.418 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:15.418 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:15.419 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:15.419 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 5' 2026-02-18T08:57:15.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:15.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:15.428 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:15.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405035 -lt 1771405089 ]] 2026-02-18T08:57:15.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 5' 2026-02-18T08:57:15.430 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:15.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:15.540 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:15.541 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:15.541 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 5' 2026-02-18T08:57:15.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:15.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:15.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:15.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405035 -lt 1771405089 ]] 2026-02-18T08:57:15.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 5' 2026-02-18T08:57:15.554 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:15.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:15.669 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:15.669 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:15.669 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 5' 2026-02-18T08:57:15.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:15.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:15.680 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:15.681 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405035 -lt 1771405089 ]] 2026-02-18T08:57:15.681 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 5' 2026-02-18T08:57:15.681 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:15.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:15.792 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:15.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 5' 2026-02-18T08:57:15.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:15.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:15.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:15.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:15.805 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405035 -lt 1771405089 ]] 2026-02-18T08:57:15.805 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 5' 2026-02-18T08:57:15.806 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:16.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:16.142 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:16.143 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 5' 2026-02-18T08:57:16.143 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:16.152 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-18T08:57:16.152 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-18T08:57:16.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-18T08:57:16.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-18T08:57:16.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-18T08:57:16.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:154: TEST_5_mons_checks: ceph mon ok-to-stop dne 2026-02-18T08:57:16.254 INFO:tasks.workunit.client.0.trial118.stderr:invalid mon(s) specified: [dne] 2026-02-18T08:57:16.264 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:155: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-18T08:57:16.372 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-18T08:57:16.380 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:156: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-18T08:57:16.482 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (a,c,d,e) after stopping [b] 2026-02-18T08:57:16.491 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:157: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-18T08:57:16.597 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (a,b,d,e) after stopping [c] 2026-02-18T08:57:16.605 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:158: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-18T08:57:16.723 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (a,b,c,e) after stopping [d] 2026-02-18T08:57:16.731 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:159: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-18T08:57:16.849 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (a,b,c,d) after stopping [e] 2026-02-18T08:57:16.857 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:160: TEST_5_mons_checks: ceph mon ok-to-stop a b 2026-02-18T08:57:16.974 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (c,d,e) after stopping [a,b] 2026-02-18T08:57:16.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:161: TEST_5_mons_checks: ceph mon ok-to-stop c d 2026-02-18T08:57:17.095 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (a,b,e) after stopping [c,d] 2026-02-18T08:57:17.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:162: TEST_5_mons_checks: ceph mon ok-to-stop a b c 2026-02-18T08:57:17.195 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (d,e) after stopping mons [a,b,c] 2026-02-18T08:57:17.199 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:164: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-18T08:57:20.305 INFO:tasks.workunit.client.0.trial118.stderr:adding another mon that is not yet online will not break quorum 2026-02-18T08:57:20.313 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:166: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-18T08:57:20.427 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.a 2026-02-18T08:57:20.435 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:167: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-18T08:57:20.547 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.b 2026-02-18T08:57:20.557 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:168: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-18T08:57:20.664 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.c 2026-02-18T08:57:20.672 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:169: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-18T08:57:20.793 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.d 2026-02-18T08:57:20.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:170: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-18T08:57:20.917 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.e 2026-02-18T08:57:20.925 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:172: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-18T08:57:20.926 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:57:20.926 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:57:20.926 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:57:20.926 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:57:20.926 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:57:21.034 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:57:21.034 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:173: TEST_5_mons_checks: wait_for_quorum 60 4 2026-02-18T08:57:21.034 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-18T08:57:21.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-18T08:57:21.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-18T08:57:21.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-18T08:57:21.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-18T08:57:21.035 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-18T08:57:21.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771405101 2026-02-18T08:57:21.036 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:21.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405041 -lt 1771405101 ]] 2026-02-18T08:57:21.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:21.037 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:21.146 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:21.146 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.147 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:21.147 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:21.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:21.157 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:21.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405041 -lt 1771405101 ]] 2026-02-18T08:57:21.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:21.158 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:21.272 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:21.272 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.273 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.273 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:21.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:21.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:21.281 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:21.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405041 -lt 1771405101 ]] 2026-02-18T08:57:21.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:21.283 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:21.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:21.400 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.400 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.401 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:21.409 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:21.409 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:21.409 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:21.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405041 -lt 1771405101 ]] 2026-02-18T08:57:21.411 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:21.411 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:21.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:21.526 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.526 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.527 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:21.536 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:21.536 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:21.536 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:21.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405041 -lt 1771405101 ]] 2026-02-18T08:57:21.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:21.538 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:21.648 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:21.648 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.649 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:21.649 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.660 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:21.660 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:21.660 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:21.661 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405041 -lt 1771405101 ]] 2026-02-18T08:57:21.661 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:21.662 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:21.775 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:21.775 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.776 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.776 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:21.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:21.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:21.787 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:21.788 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405041 -lt 1771405101 ]] 2026-02-18T08:57:21.788 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:21.789 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:21.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:21.898 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:21.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:21.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:21.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:21.909 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:21.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405041 -lt 1771405101 ]] 2026-02-18T08:57:21.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:21.911 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:22.023 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:22.023 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.024 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:22.024 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.033 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:22.033 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:22.034 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:22.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405042 -lt 1771405101 ]] 2026-02-18T08:57:22.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:22.035 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:22.146 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:22.146 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.147 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:22.147 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:22.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:22.157 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:22.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405042 -lt 1771405101 ]] 2026-02-18T08:57:22.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:22.159 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:22.273 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:22.273 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.274 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.274 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:22.284 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:22.284 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:22.285 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:22.285 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405042 -lt 1771405101 ]] 2026-02-18T08:57:22.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:22.286 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:22.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:22.401 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.401 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.402 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:22.412 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:22.412 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:22.412 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:22.413 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405042 -lt 1771405101 ]] 2026-02-18T08:57:22.413 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:22.414 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:22.527 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:22.527 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.528 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.529 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:22.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:22.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:22.538 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:22.539 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405042 -lt 1771405101 ]] 2026-02-18T08:57:22.539 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:22.539 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:22.655 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:22.655 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.656 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:22.656 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.665 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:22.665 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:22.666 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:22.666 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405042 -lt 1771405101 ]] 2026-02-18T08:57:22.667 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:22.667 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:22.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:22.782 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.783 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:22.783 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:22.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:22.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:22.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405042 -lt 1771405101 ]] 2026-02-18T08:57:22.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:22.794 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:22.906 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:22.906 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.907 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:22.907 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:22.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:22.917 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:22.917 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:22.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405042 -lt 1771405101 ]] 2026-02-18T08:57:22.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:22.918 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:23.032 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:23.032 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:23.033 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:23.033 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:23.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:23.044 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:23.044 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:23.045 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405043 -lt 1771405101 ]] 2026-02-18T08:57:23.045 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:23.045 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:23.161 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:23.161 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:23.162 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:23.162 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:23.172 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:23.172 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:23.173 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:23.174 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405043 -lt 1771405101 ]] 2026-02-18T08:57:23.174 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:23.174 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:23.288 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:23.288 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:23.289 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:23.289 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:23.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:23.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:23.298 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:23.299 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405043 -lt 1771405101 ]] 2026-02-18T08:57:23.299 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:23.300 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:23.413 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:23.413 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:23.414 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:23.415 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:23.424 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:23.425 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:23.425 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:23.426 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405043 -lt 1771405101 ]] 2026-02-18T08:57:23.426 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:23.426 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:23.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:23.544 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:23.544 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:23.544 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:23.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:23.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:23.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:23.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405043 -lt 1771405101 ]] 2026-02-18T08:57:23.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:23.555 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:26.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:26.668 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:26.669 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:26.669 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:26.677 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:26.677 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:26.678 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:26.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405046 -lt 1771405101 ]] 2026-02-18T08:57:26.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:26.679 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:26.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:26.792 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:26.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":10,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:26.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:26.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:26.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:26.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:26.805 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405046 -lt 1771405101 ]] 2026-02-18T08:57:26.805 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:26.805 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:26.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:26.920 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:26.921 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:26.921 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:26.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:26.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:26.932 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:26.933 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405046 -lt 1771405101 ]] 2026-02-18T08:57:26.933 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:26.934 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:27.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:27.047 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:27.048 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:27.048 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:27.057 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:27.057 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:27.057 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:27.058 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405047 -lt 1771405101 ]] 2026-02-18T08:57:27.058 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:27.059 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:27.169 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:27.169 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:27.170 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:27.170 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:27.180 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:27.180 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:27.181 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:27.181 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405047 -lt 1771405101 ]] 2026-02-18T08:57:27.181 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:27.182 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:27.297 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:27.297 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:27.298 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:27.298 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:27.308 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:27.308 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:27.309 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:27.310 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405047 -lt 1771405101 ]] 2026-02-18T08:57:27.310 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:27.310 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:27.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:27.422 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:27.423 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:27.423 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:27.432 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:27.433 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:27.433 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:27.434 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405047 -lt 1771405101 ]] 2026-02-18T08:57:27.434 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:27.435 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:30.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:30.551 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:30.551 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:30.551 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:30.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:30.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:30.561 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:30.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405050 -lt 1771405101 ]] 2026-02-18T08:57:30.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-18T08:57:30.563 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:42.678 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:42.679 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:42.679 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:42.680 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-18T08:57:42.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-18T08:57:42.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-18T08:57:42.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-18T08:57:42.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-18T08:57:42.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-18T08:57:42.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:175: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-18T08:57:42.790 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-18T08:57:42.798 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:176: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-18T08:57:42.913 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (c,d,e) after stopping [b] 2026-02-18T08:57:42.921 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:177: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-18T08:57:43.034 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (b,d,e) after stopping [c] 2026-02-18T08:57:43.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:178: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-18T08:57:43.157 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (b,c,e) after stopping [d] 2026-02-18T08:57:43.164 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:179: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-18T08:57:46.300 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-18T08:57:46.308 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:181: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-18T08:57:46.421 INFO:tasks.workunit.client.0.trial118.stderr:adding another mon that is not yet online will not break quorum 2026-02-18T08:57:46.429 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:183: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-18T08:57:46.544 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.a 2026-02-18T08:57:46.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:184: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-18T08:57:46.656 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.b 2026-02-18T08:57:46.667 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:185: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-18T08:57:49.773 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.c 2026-02-18T08:57:49.780 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:186: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-18T08:57:49.882 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.d 2026-02-18T08:57:49.890 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:187: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-18T08:57:50.009 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.e 2026-02-18T08:57:50.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:189: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.e 2026-02-18T08:57:50.020 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:57:50.020 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:57:50.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:57:50.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:57:50.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:57:50.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:57:50.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:190: TEST_5_mons_checks: wait_for_quorum 60 3 2026-02-18T08:57:50.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-18T08:57:50.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-18T08:57:50.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-18T08:57:50.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-18T08:57:50.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-18T08:57:50.129 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-18T08:57:50.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1771405130 2026-02-18T08:57:50.129 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:50.130 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405070 -lt 1771405130 ]] 2026-02-18T08:57:50.130 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:50.131 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:50.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:50.242 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.243 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.243 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:50.252 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:50.252 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:50.252 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:50.253 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405070 -lt 1771405130 ]] 2026-02-18T08:57:50.253 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:50.253 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:50.362 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:50.362 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.363 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.363 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:50.372 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:50.372 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:50.373 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:50.374 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405070 -lt 1771405130 ]] 2026-02-18T08:57:50.374 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:50.374 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:50.485 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:50.486 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.486 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.487 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:50.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:50.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:50.497 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:50.498 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405070 -lt 1771405130 ]] 2026-02-18T08:57:50.498 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:50.498 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:50.612 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:50.612 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.612 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:50.613 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:50.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:50.621 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:50.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405070 -lt 1771405130 ]] 2026-02-18T08:57:50.623 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:50.623 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:50.734 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:50.734 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.735 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:50.735 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:50.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:50.745 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:50.746 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405070 -lt 1771405130 ]] 2026-02-18T08:57:50.746 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:50.747 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:50.859 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:50.860 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.860 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:50.860 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:50.870 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:50.870 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:50.870 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:50.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405070 -lt 1771405130 ]] 2026-02-18T08:57:50.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:50.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:53.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:53.988 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:53.988 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:53.989 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:53.997 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:53.997 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:53.997 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:53.998 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405073 -lt 1771405130 ]] 2026-02-18T08:57:53.998 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:53.999 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:54.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:54.109 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:54.110 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:54.110 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:54.120 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:54.120 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:54.120 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:54.121 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405074 -lt 1771405130 ]] 2026-02-18T08:57:54.121 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:54.122 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:57.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:57.236 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.237 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:57.237 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.245 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:57.245 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:57.246 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:57.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405077 -lt 1771405130 ]] 2026-02-18T08:57:57.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:57.247 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:57.358 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:57.358 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.359 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:57.359 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.368 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:57.368 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:57.369 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:57.370 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405077 -lt 1771405130 ]] 2026-02-18T08:57:57.370 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:57.370 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:57.484 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:57.484 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.485 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:57.485 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:57.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:57.496 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:57.497 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405077 -lt 1771405130 ]] 2026-02-18T08:57:57.497 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:57.498 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:57.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:57.610 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.611 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:57.611 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.620 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:57.620 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:57.621 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:57.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405077 -lt 1771405130 ]] 2026-02-18T08:57:57.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:57.622 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:57.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:57.738 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.739 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:57.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:57.748 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:57.748 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:57.749 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405077 -lt 1771405130 ]] 2026-02-18T08:57:57.749 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:57.749 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:57.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:57.865 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.866 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:57.866 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":16,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:57.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:57.876 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:57.877 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405077 -lt 1771405130 ]] 2026-02-18T08:57:57.877 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:57.878 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:57.992 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:57.992 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":17,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.993 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":17,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:57.993 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:58.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:58.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:58.004 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:58.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405077 -lt 1771405130 ]] 2026-02-18T08:57:58.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:58.005 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:57:58.116 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:57:58.116 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":17,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:58.117 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":18,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":17,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:57:58.117 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:57:58.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-18T08:57:58.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-18T08:57:58.126 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-18T08:57:58.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1771405078 -lt 1771405130 ]] 2026-02-18T08:57:58.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-18T08:57:58.128 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-18T08:58:05.657 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-18T08:58:05.657 INFO:tasks.workunit.client.0.trial118.stderr:{"election_epoch":22,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:58:05.658 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-18T08:58:05.658 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":22,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"45b8fc6e-2391-45d7-88d7-7bf91afa6519","modified":"2026-02-18T08:57:08.367069Z","created":"2026-02-18T08:57:08.367069Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367061+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367066+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367067+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-18T08:57:08:367068+0000","crush_location":[]}]}}' 2026-02-18T08:58:05.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-18T08:58:05.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-18T08:58:05.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-18T08:58:05.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-18T08:58:05.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-18T08:58:05.669 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:192: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-18T08:58:08.773 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-18T08:58:08.783 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:193: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-18T08:58:11.871 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-18T08:58:11.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:194: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-18T08:58:14.964 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-18T08:58:14.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:195: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-18T08:58:21.052 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-18T08:58:21.054 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:196: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-18T08:58:24.160 INFO:tasks.workunit.client.0.trial118.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-18T08:58:24.167 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:198: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-18T08:58:24.252 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-18T08:58:24.255 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:200: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-18T08:58:24.360 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.a 2026-02-18T08:58:24.367 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:201: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-18T08:58:24.467 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: removing mon.b would break quorum 2026-02-18T08:58:24.469 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:202: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-18T08:58:24.554 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-18T08:58:24.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:203: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-18T08:58:30.643 INFO:tasks.workunit.client.0.trial118.stderr:Error EBUSY: removing mon.d would break quorum 2026-02-18T08:58:30.645 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:204: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-18T08:58:30.753 INFO:tasks.workunit.client.0.trial118.stderr:safe to remove mon.e 2026-02-18T08:58:30.761 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-18T08:58:30.762 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:30.762 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:30.762 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:30.762 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:30.762 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:30.870 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:30.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-18T08:58:30.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:58:30.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:58:30.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:58:30.871 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:30.871 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:30.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:30.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:30.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:30.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:30.877 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:58:30.878 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:58:30.878 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:58:30.880 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:58:30.880 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:58:30.880 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:58:30.881 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:30.881 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:58:30.882 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:58:30.882 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:30.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:58:30.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:58:30.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:58:30.902 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:58:30.902 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:30.902 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:58:30.902 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:58:30.904 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:58:30.904 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:58:30.904 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-18T08:58:30.905 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/ok-to-stop 0 2026-02-18T08:58:30.905 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-18T08:58:30.905 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-18T08:58:30.905 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-18T08:58:30.905 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:30.905 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:30.905 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:30.905 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:30.905 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:30.906 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:30.907 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:58:30.907 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:58:30.908 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:58:30.909 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:58:30.909 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:58:30.909 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:58:30.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:30.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:58:30.911 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:58:30.911 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:30.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:58:30.913 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-18T08:58:30.913 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-18T08:58:30.915 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:58:30.915 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:30.916 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31971 2026-02-18T08:58:30.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31971 2026-02-18T08:58:30.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:58:30.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:58:30.917 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-18T08:58:30.918 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-18T08:58:30.918 DEBUG:teuthology.orchestra.run.trial118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-18T08:58:30.968 INFO:tasks.workunit:Running workunit misc/rados-striper.sh... 2026-02-18T08:58:30.969 DEBUG:teuthology.orchestra.run.trial118:workunit test misc/rados-striper.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh 2026-02-18T08:58:31.022 INFO:tasks.workunit.client.0.trial118.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-18T08:58:31.025 INFO:tasks.workunit.client.0.trial118.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-18T08:58:31.025 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-18T08:58:31.025 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-18T08:58:31.025 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-18T08:58:31.025 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-18T08:58:31.025 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-18T08:58:31.025 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-18T08:58:31.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-18T08:58:31.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-18T08:58:31.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/rados-striper 2026-02-18T08:58:31.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:20: run: local dir=td/rados-striper 2026-02-18T08:58:31.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:21: run: shift 2026-02-18T08:58:31.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: export CEPH_MON=127.0.0.1:7116 2026-02-18T08:58:31.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: CEPH_MON=127.0.0.1:7116 2026-02-18T08:58:31.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:24: run: export CEPH_ARGS 2026-02-18T08:58:31.026 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: uuidgen 2026-02-18T08:58:31.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: CEPH_ARGS+='--fsid=c0da68ca-fd10-4686-836e-5c5ad1454c8a --auth-supported=none ' 2026-02-18T08:58:31.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:26: run: CEPH_ARGS+='--mon-host=127.0.0.1:7116 ' 2026-02-18T08:58:31.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:29: run: setup td/rados-striper 2026-02-18T08:58:31.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/rados-striper 2026-02-18T08:58:31.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/rados-striper 2026-02-18T08:58:31.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-18T08:58:31.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:58:31.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-18T08:58:31.029 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:31.029 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:31.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:31.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:31.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:31.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:31.030 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:58:31.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:58:31.030 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:58:31.031 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:58:31.031 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:58:31.031 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:58:31.032 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:31.032 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:58:31.033 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:58:31.033 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:31.033 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:58:31.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:58:31.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-18T08:58:31.036 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:58:31.036 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:31.036 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.48105 2026-02-18T08:58:31.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.48105 2026-02-18T08:58:31.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:58:31.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:58:31.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/rados-striper 2026-02-18T08:58:31.039 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:58:31.039 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:31.039 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.48105 2026-02-18T08:58:31.040 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.48105 2026-02-18T08:58:31.041 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:58:31.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-18T08:58:31.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-18T08:58:31.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:58:31.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/rados-striper 1' TERM HUP INT 2026-02-18T08:58:31.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:32: run: run_mon td/rados-striper a 2026-02-18T08:58:31.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/rados-striper 2026-02-18T08:58:31.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:58:31.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:58:31.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:58:31.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/rados-striper/a 2026-02-18T08:58:31.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/rados-striper/a --run-dir=td/rados-striper 2026-02-18T08:58:31.066 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:58:31.066 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:31.066 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:31.066 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:31.066 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:31.066 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.48105 2026-02-18T08:58:31.067 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.48105/$cluster-$name.asok' 2026-02-18T08:58:31.067 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/rados-striper/a '--log-file=td/rados-striper/$name.log' '--admin-socket=/tmp/ceph-asok.48105/$cluster-$name.asok' --mon-cluster-log-file=td/rados-striper/log --run-dir=td/rados-striper '--pid-file=td/rados-striper/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:58:31.093 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:58:31.094 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:58:31.094 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:58:31.095 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:58:31.095 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:58:31.096 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:58:31.096 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:58:31.096 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:58:31.096 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:58:31.096 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:31.096 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:31.097 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.48105 2026-02-18T08:58:31.097 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.48105/ceph-mon.a.asok 2026-02-18T08:58:31.097 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:58:31.097 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.48105/ceph-mon.a.asok config get fsid 2026-02-18T08:58:31.150 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:58:31.151 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:58:31.151 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:58:31.151 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:58:31.151 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:58:31.152 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:58:31.152 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:58:31.152 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:58:31.152 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:31.152 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:31.152 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.48105 2026-02-18T08:58:31.152 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.48105/ceph-mon.a.asok 2026-02-18T08:58:31.153 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:58:31.153 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.48105/ceph-mon.a.asok config get mon_host 2026-02-18T08:58:31.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:33: run: run_osd td/rados-striper 0 2026-02-18T08:58:31.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-18T08:58:31.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:58:31.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/0 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c0da68ca-fd10-4686-836e-5c5ad1454c8a --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/0' 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/0/journal' 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-18T08:58:31.209 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.48105 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.48105/$cluster-$name.asok' 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.48105/$cluster-$name.asok' 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-18T08:58:31.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:58:31.211 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:58:31.211 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:58:31.211 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:58:31.211 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:58:31.211 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:58:31.211 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/0 2026-02-18T08:58:31.212 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:58:31.212 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0aeabfe6-9aa7-4bad-acc5-24a09e61163c 2026-02-18T08:58:31.212 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0aeabfe6-9aa7-4bad-acc5-24a09e61163c' 2026-02-18T08:58:31.212 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 0aeabfe6-9aa7-4bad-acc5-24a09e61163c 2026-02-18T08:58:31.213 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:58:31.220 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA3f5VpsIB5DBAAPILUnZPdD9hB2vlUJpf0jQ== 2026-02-18T08:58:31.220 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA3f5VpsIB5DBAAPILUnZPdD9hB2vlUJpf0jQ=="}' 2026-02-18T08:58:31.220 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0aeabfe6-9aa7-4bad-acc5-24a09e61163c -i td/rados-striper/0/new.json 2026-02-18T08:58:31.339 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:31.349 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/0/new.json 2026-02-18T08:58:31.350 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=c0da68ca-fd10-4686-836e-5c5ad1454c8a --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.48105/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA3f5VpsIB5DBAAPILUnZPdD9hB2vlUJpf0jQ== --osd-uuid 0aeabfe6-9aa7-4bad-acc5-24a09e61163c 2026-02-18T08:58:31.374 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:31.360+0000 7f97e5ee6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:31.375 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:31.360+0000 7f97e5ee6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:31.377 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:31.364+0000 7f97e5ee6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:31.377 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:31.364+0000 7f97e5ee6a40 -1 bdev(0x555cd8880400 td/rados-striper/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:58:31.378 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:31.364+0000 7f97e5ee6a40 -1 bluestore(td/rados-striper/0) _read_fsid unparsable uuid 2026-02-18T08:58:32.018 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/0/keyring 2026-02-18T08:58:32.018 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:58:32.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:58:32.019 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:58:32.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:58:32.137 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:58:32.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:58:32.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=c0da68ca-fd10-4686-836e-5c5ad1454c8a --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.48105/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:58:32.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:58:32.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:58:32.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:58:32.159 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:32.144+0000 7f0b77c38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:32.161 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:32.148+0000 7f0b77c38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:32.162 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:32.148+0000 7f0b77c38a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:32.246 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:58:32.246 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:58:32.246 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:58:32.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:58:32.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:58:32.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:32.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:58:32.247 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:32.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:32.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:32.358 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:32.550 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:32.536+0000 7f0b77c38a40 -1 Falling back to public interface 2026-02-18T08:58:33.207 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:33.192+0000 7f0b77c38a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:58:33.360 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:33.360 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:33.361 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:58:33.361 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:58:33.361 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:33.361 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:33.472 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:34.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:34.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:34.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:58:34.474 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:58:34.475 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:34.475 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:34.580 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1651983347,v1:127.0.0.1:6801/1651983347] [v2:127.0.0.1:6802/1651983347,v1:127.0.0.1:6803/1651983347] exists,up 0aeabfe6-9aa7-4bad-acc5-24a09e61163c 2026-02-18T08:58:34.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:58:34.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:58:34.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:58:34.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:34: run: run_osd td/rados-striper 1 2026-02-18T08:58:34.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-18T08:58:34.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:58:34.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-18T08:58:34.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:58:34.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/1 2026-02-18T08:58:34.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c0da68ca-fd10-4686-836e-5c5ad1454c8a --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-18T08:58:34.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:58:34.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:58:34.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:58:34.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/1' 2026-02-18T08:58:34.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/1/journal' 2026-02-18T08:58:34.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:58:34.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:58:34.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-18T08:58:34.585 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.48105 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.48105/$cluster-$name.asok' 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.48105/$cluster-$name.asok' 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-18T08:58:34.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:58:34.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:58:34.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:58:34.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:58:34.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:58:34.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:58:34.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/1 2026-02-18T08:58:34.587 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:58:34.588 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bce146e9-c61d-4814-9c94-19ebef65303a 2026-02-18T08:58:34.588 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 bce146e9-c61d-4814-9c94-19ebef65303a' 2026-02-18T08:58:34.588 INFO:tasks.workunit.client.0.trial118.stdout:add osd1 bce146e9-c61d-4814-9c94-19ebef65303a 2026-02-18T08:58:34.588 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:58:34.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA6f5VpO2zeIhAAUEGEjpERawS+zezP9Zvsgg== 2026-02-18T08:58:34.596 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA6f5VpO2zeIhAAUEGEjpERawS+zezP9Zvsgg=="}' 2026-02-18T08:58:34.596 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bce146e9-c61d-4814-9c94-19ebef65303a -i td/rados-striper/1/new.json 2026-02-18T08:58:34.714 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:58:34.722 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/1/new.json 2026-02-18T08:58:34.722 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=c0da68ca-fd10-4686-836e-5c5ad1454c8a --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.48105/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA6f5VpO2zeIhAAUEGEjpERawS+zezP9Zvsgg== --osd-uuid bce146e9-c61d-4814-9c94-19ebef65303a 2026-02-18T08:58:34.737 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:34.724+0000 7f6d68287a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:34.739 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:34.724+0000 7f6d68287a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:34.740 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:34.724+0000 7f6d68287a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:34.740 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:34.724+0000 7f6d68287a40 -1 bdev(0x55747d970400 td/rados-striper/1/block) open stat got: (1) Operation not permitted 2026-02-18T08:58:34.741 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:34.724+0000 7f6d68287a40 -1 bluestore(td/rados-striper/1) _read_fsid unparsable uuid 2026-02-18T08:58:35.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/1/keyring 2026-02-18T08:58:35.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:58:35.353 INFO:tasks.workunit.client.0.trial118.stdout:adding osd1 key to auth repository 2026-02-18T08:58:35.353 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-18T08:58:35.353 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:58:35.472 INFO:tasks.workunit.client.0.trial118.stdout:start osd.1 2026-02-18T08:58:35.473 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-18T08:58:35.473 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=c0da68ca-fd10-4686-836e-5c5ad1454c8a --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.48105/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:58:35.473 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:58:35.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:58:35.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:58:35.496 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:35.480+0000 7f48d7d2ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:35.498 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:35.484+0000 7f48d7d2ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:35.499 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:35.484+0000 7f48d7d2ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:35.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-18T08:58:35.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:58:35.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-18T08:58:35.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:58:35.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:58:35.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:35.585 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:35.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:58:35.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:35.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:58:35.691 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:35.890 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:35.876+0000 7f48d7d2ba40 -1 Falling back to public interface 2026-02-18T08:58:36.518 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:36.504+0000 7f48d7d2ba40 -1 osd.1 0 log_to_monitors true 2026-02-18T08:58:36.692 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:36.692 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:36.693 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:58:36.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:58:36.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:36.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:58:36.798 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:37.799 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:37.800 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:37.800 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:58:37.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:58:37.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:37.802 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:58:37.906 INFO:tasks.workunit.client.0.trial118.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/1288338630,v1:127.0.0.1:6809/1288338630] [v2:127.0.0.1:6810/1288338630,v1:127.0.0.1:6811/1288338630] exists,up bce146e9-c61d-4814-9c94-19ebef65303a 2026-02-18T08:58:37.909 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:58:37.909 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:58:37.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:58:37.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:35: run: run_osd td/rados-striper 2 2026-02-18T08:58:37.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-18T08:58:37.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:58:37.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-18T08:58:37.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:58:37.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/2 2026-02-18T08:58:37.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c0da68ca-fd10-4686-836e-5c5ad1454c8a --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-18T08:58:37.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:58:37.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:58:37.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/2' 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/2/journal' 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.48105 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.48105/$cluster-$name.asok' 2026-02-18T08:58:37.911 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.48105/$cluster-$name.asok' 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:58:37.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/2 2026-02-18T08:58:37.913 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:58:37.913 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d390c9fd-ae24-4d97-b935-008d953e7835 2026-02-18T08:58:37.913 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d390c9fd-ae24-4d97-b935-008d953e7835' 2026-02-18T08:58:37.914 INFO:tasks.workunit.client.0.trial118.stdout:add osd2 d390c9fd-ae24-4d97-b935-008d953e7835 2026-02-18T08:58:37.914 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:58:37.923 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA9f5Vp8ARjNhAAslylwVPikX42IAZVyKPPdw== 2026-02-18T08:58:37.923 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA9f5Vp8ARjNhAAslylwVPikX42IAZVyKPPdw=="}' 2026-02-18T08:58:37.923 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d390c9fd-ae24-4d97-b935-008d953e7835 -i td/rados-striper/2/new.json 2026-02-18T08:58:38.032 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:58:38.039 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/2/new.json 2026-02-18T08:58:38.040 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=c0da68ca-fd10-4686-836e-5c5ad1454c8a --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.48105/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA9f5Vp8ARjNhAAslylwVPikX42IAZVyKPPdw== --osd-uuid d390c9fd-ae24-4d97-b935-008d953e7835 2026-02-18T08:58:38.056 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:38.040+0000 7fb521faaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:38.057 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:38.044+0000 7fb521faaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:38.059 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:38.044+0000 7fb521faaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:38.059 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:38.044+0000 7fb521faaa40 -1 bdev(0x55627e6c4400 td/rados-striper/2/block) open stat got: (1) Operation not permitted 2026-02-18T08:58:38.060 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:38.044+0000 7fb521faaa40 -1 bluestore(td/rados-striper/2) _read_fsid unparsable uuid 2026-02-18T08:58:38.676 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/2/keyring 2026-02-18T08:58:38.676 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:58:38.677 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-18T08:58:38.677 INFO:tasks.workunit.client.0.trial118.stdout:adding osd2 key to auth repository 2026-02-18T08:58:38.678 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:58:38.800 INFO:tasks.workunit.client.0.trial118.stdout:start osd.2 2026-02-18T08:58:38.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-18T08:58:38.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=c0da68ca-fd10-4686-836e-5c5ad1454c8a --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.48105/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:58:38.802 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:58:38.802 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:58:38.802 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:58:38.815 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:38.800+0000 7fbab002da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:38.817 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:38.804+0000 7fbab002da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:38.818 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:38.804+0000 7fbab002da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:38.909 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-18T08:58:38.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:58:38.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-18T08:58:38.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:58:38.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:58:38.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:38.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:58:38.910 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:38.911 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:38.911 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-18T08:58:39.017 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:39.218 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:39.204+0000 7fbab002da40 -1 Falling back to public interface 2026-02-18T08:58:39.809 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:39.796+0000 7fbab002da40 -1 osd.2 0 log_to_monitors true 2026-02-18T08:58:40.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:40.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:40.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:58:40.020 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:58:40.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:40.021 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-18T08:58:40.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:41.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:41.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:41.129 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:58:41.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:58:41.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:41.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-18T08:58:41.234 INFO:tasks.workunit.client.0.trial118.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/2203788751,v1:127.0.0.1:6817/2203788751] [v2:127.0.0.1:6818/2203788751,v1:127.0.0.1:6819/2203788751] exists,up d390c9fd-ae24-4d97-b935-008d953e7835 2026-02-18T08:58:41.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:58:41.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:58:41.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:58:41.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:36: run: create_rbd_pool 2026-02-18T08:58:41.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T08:58:41.333 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T08:58:41.341 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T08:58:41.341 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T08:58:41.481 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T08:58:41.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T08:58:42.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T08:58:42.769 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:39: run: dd if=/dev/urandom of=td/rados-striper/toyfile bs=1234 count=1 2026-02-18T08:58:42.770 INFO:tasks.workunit.client.0.trial118.stderr:1+0 records in 2026-02-18T08:58:42.770 INFO:tasks.workunit.client.0.trial118.stderr:1+0 records out 2026-02-18T08:58:42.770 INFO:tasks.workunit.client.0.trial118.stderr:1234 bytes (1.2 kB, 1.2 KiB) copied, 5.7667e-05 s, 21.4 MB/s 2026-02-18T08:58:42.770 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:42: run: rados --pool rbd --striper put toyfile td/rados-striper/toyfile 2026-02-18T08:58:42.798 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: rados --pool rbd --striper stat toyfile 2026-02-18T08:58:42.798 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: cut -d , -f 2 2026-02-18T08:58:42.822 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-18T08:58:42.822 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: cut -d , -f 2 2026-02-18T08:58:42.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:47: run: echo ' size 1234' 2026-02-18T08:58:42.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:48: run: diff -w td/rados-striper/stripedStat td/rados-striper/refstat 2026-02-18T08:58:42.847 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:49: run: diff -w td/rados-striper/stat td/rados-striper/refstat 2026-02-18T08:58:42.848 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:50: run: rados --pool rbd stat toyfile 2026-02-18T08:58:42.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:51: run: grep -q 'No such file or directory' td/rados-striper/staterror 2026-02-18T08:58:42.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:54: run: rados --pool rbd --striper get toyfile td/rados-striper/stripedGroup 2026-02-18T08:58:42.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:55: run: diff -w td/rados-striper/toyfile td/rados-striper/stripedGroup 2026-02-18T08:58:42.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:56: run: rados --pool rbd get toyfile.0000000000000000 td/rados-striper/nonSTripedGroup 2026-02-18T08:58:42.915 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:57: run: diff -w td/rados-striper/toyfile td/rados-striper/nonSTripedGroup 2026-02-18T08:58:42.915 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:60: run: rados --pool rbd --striper truncate toyfile 12 2026-02-18T08:58:42.936 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: rados --pool rbd --striper stat toyfile 2026-02-18T08:58:42.936 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: cut -d , -f 2 2026-02-18T08:58:42.960 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-18T08:58:42.960 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: cut -d , -f 2 2026-02-18T08:58:42.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:63: run: echo ' size 12' 2026-02-18T08:58:42.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:64: run: diff -w td/rados-striper/stripedStat td/rados-striper/reftrunc 2026-02-18T08:58:42.982 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:65: run: diff -w td/rados-striper/stat td/rados-striper/reftrunc 2026-02-18T08:58:42.983 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:69: run: rados --pool rbd --striper setxattr toyfile somexattr somevalue 2026-02-18T08:58:43.009 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:70: run: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-18T08:58:43.033 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:71: run: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-18T08:58:43.057 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:72: run: echo somevalue 2026-02-18T08:58:43.057 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:73: run: diff -w td/rados-striper/xattrvalue td/rados-striper/refvalue 2026-02-18T08:58:43.057 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:74: run: diff -w td/rados-striper/xattrvalue2 td/rados-striper/refvalue 2026-02-18T08:58:43.058 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:75: run: rados --pool rbd --striper listxattr toyfile 2026-02-18T08:58:43.079 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:76: run: echo somexattr 2026-02-18T08:58:43.079 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:77: run: diff -w td/rados-striper/xattrlist td/rados-striper/reflist 2026-02-18T08:58:43.080 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: rados --pool rbd listxattr toyfile.0000000000000000 2026-02-18T08:58:43.080 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: grep -v striper 2026-02-18T08:58:43.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:79: run: diff -w td/rados-striper/xattrlist2 td/rados-striper/reflist 2026-02-18T08:58:43.100 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:80: run: rados --pool rbd --striper rmxattr toyfile somexattr 2026-02-18T08:58:43.122 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:82: run: local 'attr_not_found_str=No data available' 2026-02-18T08:58:43.123 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: uname 2026-02-18T08:58:43.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: '[' Linux = FreeBSD ']' 2026-02-18T08:58:43.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:85: run: expect_failure td/rados-striper 'No data available' rados --pool rbd --striper getxattr toyfile somexattr 2026-02-18T08:58:43.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-18T08:58:43.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-18T08:58:43.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-18T08:58:43.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-18T08:58:43.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-18T08:58:43.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-18T08:58:43.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-18T08:58:43.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-18T08:58:43.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-18T08:58:43.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-18T08:58:43.144 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:87: run: expect_failure td/rados-striper 'No data available' rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-18T08:58:43.144 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-18T08:58:43.144 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-18T08:58:43.144 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-18T08:58:43.144 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-18T08:58:43.144 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-18T08:58:43.144 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-18T08:58:43.161 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-18T08:58:43.161 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-18T08:58:43.161 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-18T08:58:43.162 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-18T08:58:43.162 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:91: run: rados --pool rbd --striper rm toyfile 2026-02-18T08:58:43.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:92: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd --striper stat toyfile 2026-02-18T08:58:43.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-18T08:58:43.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-18T08:58:43.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-18T08:58:43.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-18T08:58:43.184 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-18T08:58:43.184 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper stat toyfile 2026-02-18T08:58:43.206 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-18T08:58:43.206 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-18T08:58:43.206 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-18T08:58:43.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-18T08:58:43.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:94: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd stat toyfile.0000000000000000 2026-02-18T08:58:43.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-18T08:58:43.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-18T08:58:43.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-18T08:58:43.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-18T08:58:43.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-18T08:58:43.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd stat toyfile.0000000000000000 2026-02-18T08:58:43.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-18T08:58:43.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-18T08:58:43.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-18T08:58:43.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-18T08:58:43.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:98: run: teardown td/rados-striper 2026-02-18T08:58:43.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-18T08:58:43.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:58:43.225 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-18T08:58:43.225 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:43.225 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:43.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:43.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:43.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:43.333 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:43.334 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:58:43.334 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:58:43.335 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:58:43.336 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:58:43.336 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:58:43.337 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:58:43.337 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:43.338 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:58:43.338 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:58:43.338 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:43.339 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:58:43.340 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:58:43.340 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-18T08:58:43.349 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:58:43.349 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:43.349 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.48105 2026-02-18T08:58:43.349 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.48105 2026-02-18T08:58:43.351 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:58:43.351 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:58:43.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-18T08:58:43.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/rados-striper 0 2026-02-18T08:58:43.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-18T08:58:43.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-18T08:58:43.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-18T08:58:43.352 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:43.352 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:43.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:43.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:43.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:43.353 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:43.354 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:58:43.355 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:58:43.355 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:58:43.356 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:58:43.356 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:58:43.357 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:58:43.358 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:43.358 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:58:43.358 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:58:43.358 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:43.359 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:58:43.361 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-18T08:58:43.361 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-18T08:58:43.362 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:58:43.362 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:43.362 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.48105 2026-02-18T08:58:43.362 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.48105 2026-02-18T08:58:43.363 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:58:43.363 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:58:43.363 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-18T08:58:43.364 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-18T08:58:43.365 DEBUG:teuthology.orchestra.run.trial118:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-18T08:58:43.417 INFO:tasks.workunit:Running workunit misc/test-ceph-helpers.sh... 2026-02-18T08:58:43.417 DEBUG:teuthology.orchestra.run.trial118:workunit test misc/test-ceph-helpers.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh 2026-02-18T08:58:43.471 INFO:tasks.workunit.client.0.trial118.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-18T08:58:43.474 INFO:tasks.workunit.client.0.trial118.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-18T08:58:43.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2414: run_tests: export PATH=./bin:.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-18T08:58:43.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2414: run_tests: PATH=./bin:.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-18T08:58:43.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2416: run_tests: export CEPH_MON=127.0.0.1:7109 2026-02-18T08:58:43.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2416: run_tests: CEPH_MON=127.0.0.1:7109 2026-02-18T08:58:43.475 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2417: run_tests: export CEPH_ARGS 2026-02-18T08:58:43.475 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2418: run_tests: uuidgen 2026-02-18T08:58:43.476 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2418: run_tests: CEPH_ARGS+=' --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none ' 2026-02-18T08:58:43.476 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2419: run_tests: CEPH_ARGS+='--mon-host=127.0.0.1:7109 ' 2026-02-18T08:58:43.476 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2420: run_tests: export CEPH_CONF=/dev/null 2026-02-18T08:58:43.476 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2420: run_tests: CEPH_CONF=/dev/null 2026-02-18T08:58:43.476 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: set 2026-02-18T08:58:43.476 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: sed -n -e 's/^\(test_[0-9a-z_]*\) .*/\1/p' 2026-02-18T08:58:43.478 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: local 'funcs=test_activate_osd 2026-02-18T08:58:43.478 INFO:tasks.workunit.client.0.trial118.stderr:test_activate_osd_after_mark_down 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_activate_osd_skip_benchmark 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_destroy_osd 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_display_logs 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_erasure_code_plugin_exists 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_expect_failure 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_file_with_random_data 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_flush_pg_stats 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_get_config 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_get_is_making_recovery_progress 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_get_last_scrub_stamp 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_get_not_primary 2026-02-18T08:58:43.479 INFO:tasks.workunit.client.0.trial118.stderr:test_get_num_active_clean 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_get_num_active_or_peered 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_get_num_pgs 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_get_op_scheduler 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_get_osd_id_used_by_pgs 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_get_osds 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_get_pg 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_get_primary 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_get_timeout_delays 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_is_clean 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_kill_daemon 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_kill_daemons 2026-02-18T08:58:43.480 INFO:tasks.workunit.client.0.trial118.stderr:test_objectstore_tool 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_pg_schedule_scrub 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_pg_scrub 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_repair 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_run_in_background 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_run_mon 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_run_osd 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_set_config 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_setup 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_teardown 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_wait_background 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_wait_for_clean 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_wait_for_health_ok 2026-02-18T08:58:43.481 INFO:tasks.workunit.client.0.trial118.stderr:test_wait_for_osd 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:test_wait_for_peered 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:test_wait_for_scrub 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:test_wait_osd_id_used_by_pgs' 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: run_tests: local dir=td/ceph-helpers 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd td/ceph-helpers 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:877: test_activate_osd: local dir=td/ceph-helpers 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: test_activate_osd: setup td/ceph-helpers 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:58:43.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:58:43.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:58:43.483 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:43.483 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:43.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:43.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:43.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:43.484 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:43.484 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:58:43.485 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:58:43.485 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:58:43.485 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:58:43.485 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:58:43.485 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:58:43.485 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:43.485 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:58:43.485 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:58:43.485 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:43.486 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:58:43.486 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:58:43.487 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:58:43.488 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:58:43.488 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:43.488 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:43.488 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:58:43.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:58:43.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:58:43.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:58:43.491 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:58:43.491 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:43.491 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:43.491 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:58:43.493 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:58:43.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-18T08:58:43.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-18T08:58:43.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:58:43.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:58:43.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:881: test_activate_osd: run_mon td/ceph-helpers a 2026-02-18T08:58:43.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:58:43.495 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:58:43.495 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:58:43.495 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:58:43.495 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:58:43.495 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-18T08:58:43.518 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:58:43.518 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:43.519 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:43.519 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:43.519 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:43.519 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:43.519 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:43.519 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:58:43.548 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:58:43.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:58:43.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:58:43.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:58:43.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:58:43.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:58:43.551 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:58:43.551 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:58:43.551 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:58:43.551 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:43.551 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:43.551 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:43.551 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:58:43.551 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:58:43.551 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:58:43.605 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:58:43.605 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:58:43.605 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:58:43.605 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:58:43.606 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:58:43.606 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:58:43.606 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:58:43.606 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:58:43.606 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:43.607 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:43.607 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:43.607 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:58:43.607 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:58:43.607 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:58:43.659 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: test_activate_osd: run_mgr td/ceph-helpers x 2026-02-18T08:58:43.660 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:58:43.660 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:58:43.660 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:58:43.660 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:58:43.660 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:58:43.660 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:58:43.769 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:58:43.769 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:43.769 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:43.769 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:43.769 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:43.770 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:43.770 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:43.770 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:58:43.771 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:58:43.791 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:884: test_activate_osd: run_osd td/ceph-helpers 0 2026-02-18T08:58:43.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T08:58:43.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:58:43.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:58:43.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:58:43.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:58:43.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:58:43.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:58:43.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:58:43.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:58:43.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:58:43.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:43.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:58:43.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:58:43.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:58:43.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:58:43.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:58:43.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:58:43.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:58:43.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:58:43.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:58:43.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:58:43.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:58:43.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:58:43.795 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:58:43.795 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=baf4bf6a-bb59-4cfa-8ced-08710b3396eb 2026-02-18T08:58:43.795 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 baf4bf6a-bb59-4cfa-8ced-08710b3396eb' 2026-02-18T08:58:43.796 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 baf4bf6a-bb59-4cfa-8ced-08710b3396eb 2026-02-18T08:58:43.796 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:58:43.805 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBDf5VpnDxfLxAA6cpMbcY0SJFYmQL+nBxaWA== 2026-02-18T08:58:43.805 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBDf5VpnDxfLxAA6cpMbcY0SJFYmQL+nBxaWA=="}' 2026-02-18T08:58:43.805 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new baf4bf6a-bb59-4cfa-8ced-08710b3396eb -i td/ceph-helpers/0/new.json 2026-02-18T08:58:43.917 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:43.925 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T08:58:43.926 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBDf5VpnDxfLxAA6cpMbcY0SJFYmQL+nBxaWA== --osd-uuid baf4bf6a-bb59-4cfa-8ced-08710b3396eb 2026-02-18T08:58:43.943 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:43.927+0000 7f972c542a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:43.944 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:43.927+0000 7f972c542a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:43.945 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:43.931+0000 7f972c542a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:43.946 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:43.931+0000 7f972c542a40 -1 bdev(0x5613f9008400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:58:43.946 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:43.931+0000 7f972c542a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T08:58:44.836 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T08:58:44.836 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:58:44.836 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:58:44.837 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:58:44.837 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:58:44.955 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:58:44.956 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:58:44.956 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:58:44.957 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:58:44.957 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:58:44.957 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:58:44.979 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:44.963+0000 7f9f1da8da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:44.982 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:44.967+0000 7f9f1da8da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:44.983 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:44.967+0000 7f9f1da8da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:45.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:58:45.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:58:45.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:58:45.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:58:45.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:58:45.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:45.065 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:45.065 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:58:45.065 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:45.065 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:45.170 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:45.518 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:45.503+0000 7f9f1da8da40 -1 Falling back to public interface 2026-02-18T08:58:46.171 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:46.171 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:46.172 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:58:46.172 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:58:46.173 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:46.173 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:46.382 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:46.421 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:46.407+0000 7f9f1da8da40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:58:47.384 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:47.384 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:47.384 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:58:47.385 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:58:47.385 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:47.385 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:47.594 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2894104261,v1:127.0.0.1:6801/2894104261] [v2:127.0.0.1:6802/2894104261,v1:127.0.0.1:6803/2894104261] exists,up baf4bf6a-bb59-4cfa-8ced-08710b3396eb 2026-02-18T08:58:47.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:58:47.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:58:47.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:58:47.595 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: get_asok_path osd.0 2026-02-18T08:58:47.595 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:58:47.596 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:58:47.596 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:47.596 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:47.596 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:47.596 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-osd.0.asok 2026-02-18T08:58:47.596 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: CEPH_ARGS= 2026-02-18T08:58:47.596 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.53296/ceph-osd.0.asok config get osd_max_backfills 2026-02-18T08:58:47.652 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: local 'backfills={"osd_max_backfills":"1"}' 2026-02-18T08:58:47.653 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: echo '{"osd_max_backfills":"1"}' 2026-02-18T08:58:47.653 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: grep --quiet osd_max_backfills 2026-02-18T08:58:47.654 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:889: test_activate_osd: kill_daemons td/ceph-helpers TERM osd 2026-02-18T08:58:47.655 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:47.655 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:47.655 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:47.655 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:47.655 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:47.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:47.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:891: test_activate_osd: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-18T08:58:47.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-18T08:58:47.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:58:47.966 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:58:47.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:58:47.968 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:58:47.968 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-18T08:58:47.968 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-18T08:58:47.968 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:58:47.968 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:58:47.969 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-18T08:58:47.969 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-18T08:58:47.969 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-18T08:58:47.970 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-18T08:58:47.970 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-18T08:58:47.970 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-18T08:58:47.970 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-18T08:58:47.990 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:47.975+0000 7f425f68ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:47.992 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:47.975+0000 7f425f68ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:47.993 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:47.979+0000 7f425f68ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:48.186 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-18T08:58:48.186 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:58:48.186 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:58:48.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:58:48.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:58:48.187 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:48.187 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:48.188 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:58:48.188 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:48.188 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:48.394 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:48.379+0000 7f425f68ba40 -1 Falling back to public interface 2026-02-18T08:58:48.404 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:49.004 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:48.987+0000 7f425f68ba40 -1 osd.0 5 log_to_monitors true 2026-02-18T08:58:49.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:49.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:49.406 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:58:49.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:58:49.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:49.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:49.619 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/1104876561,v1:127.0.0.1:6801/1104876561] [v2:127.0.0.1:6802/1104876561,v1:127.0.0.1:6803/1104876561] exists,up baf4bf6a-bb59-4cfa-8ced-08710b3396eb 2026-02-18T08:58:49.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:58:49.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:58:49.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:58:49.622 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: get_asok_path osd.0 2026-02-18T08:58:49.622 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:58:49.622 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:58:49.622 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:49.622 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:49.622 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:49.622 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-osd.0.asok 2026-02-18T08:58:49.622 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: CEPH_ARGS= 2026-02-18T08:58:49.622 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.53296/ceph-osd.0.asok config get osd_max_backfills 2026-02-18T08:58:49.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: local 'backfills={"osd_max_backfills":"20"}' 2026-02-18T08:58:49.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:894: test_activate_osd: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-18T08:58:49.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:896: test_activate_osd: teardown td/ceph-helpers 2026-02-18T08:58:49.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:58:49.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:58:49.680 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:58:49.681 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:49.681 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:49.681 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:49.681 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:49.681 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:49.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:49.787 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:58:49.788 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:58:49.789 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:58:49.790 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:58:49.790 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:58:49.790 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:58:49.791 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:49.791 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:58:49.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:58:49.792 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:49.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:58:49.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:58:49.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:58:49.797 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:58:49.797 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:49.798 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:49.798 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:58:49.800 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:58:49.800 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:58:49.800 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:58:49.800 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd_after_mark_down td/ceph-helpers 2026-02-18T08:58:49.800 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:900: test_activate_osd_after_mark_down: local dir=td/ceph-helpers 2026-02-18T08:58:49.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:902: test_activate_osd_after_mark_down: setup td/ceph-helpers 2026-02-18T08:58:49.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:58:49.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:58:49.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:58:49.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:58:49.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:58:49.801 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:49.801 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:49.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:49.802 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:49.802 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:49.802 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:49.803 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:58:49.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:58:49.803 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:58:49.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:58:49.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:58:49.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:58:49.805 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:49.805 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:58:49.806 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:58:49.806 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:49.807 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:58:49.808 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:58:49.808 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:58:49.809 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:58:49.809 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:49.809 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:49.809 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:58:49.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:58:49.811 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:58:49.811 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:58:49.812 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:58:49.812 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:49.812 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:49.812 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:58:49.813 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:58:49.814 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:58:49.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:58:49.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:58:49.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:904: test_activate_osd_after_mark_down: run_mon td/ceph-helpers a 2026-02-18T08:58:49.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:58:49.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:58:49.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:58:49.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:58:49.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:58:49.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-18T08:58:49.836 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:58:49.836 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:49.836 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:49.836 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:49.836 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:49.836 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:49.837 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:49.837 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:58:49.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:58:49.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:58:49.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:58:49.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:58:49.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:58:49.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:58:49.865 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:58:49.865 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:58:49.865 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:58:49.866 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:49.866 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:49.867 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:49.867 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:58:49.867 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:58:49.867 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:58:49.921 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:58:49.921 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:58:49.921 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:58:49.921 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:58:49.921 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:58:49.921 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:58:49.921 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:58:49.921 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:58:49.922 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:49.923 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:49.923 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:49.923 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:58:49.923 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:58:49.923 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:58:49.975 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:905: test_activate_osd_after_mark_down: run_mgr td/ceph-helpers x 2026-02-18T08:58:49.976 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:58:49.976 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:58:49.976 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:58:49.976 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:58:49.976 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:58:49.976 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:58:50.086 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:58:50.087 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:50.087 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:50.087 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:50.087 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:50.087 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:50.088 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:50.088 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:58:50.088 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:58:50.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:907: test_activate_osd_after_mark_down: run_osd td/ceph-helpers 0 2026-02-18T08:58:50.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:58:50.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:58:50.108 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:58:50.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:58:50.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:58:50.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:58:50.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:58:50.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:58:50.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:58:50.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:58:50.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:58:50.110 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:58:50.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7809c935-1f4b-4a4e-82ed-f517b44da622 2026-02-18T08:58:50.110 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 7809c935-1f4b-4a4e-82ed-f517b44da622 2026-02-18T08:58:50.111 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7809c935-1f4b-4a4e-82ed-f517b44da622' 2026-02-18T08:58:50.111 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:58:50.119 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBKf5VpDMl3BhAA1l4KjVNDFlyJ2xK2Uco7Eg== 2026-02-18T08:58:50.119 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBKf5VpDMl3BhAA1l4KjVNDFlyJ2xK2Uco7Eg=="}' 2026-02-18T08:58:50.119 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7809c935-1f4b-4a4e-82ed-f517b44da622 -i td/ceph-helpers/0/new.json 2026-02-18T08:58:50.234 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:50.243 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T08:58:50.244 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBKf5VpDMl3BhAA1l4KjVNDFlyJ2xK2Uco7Eg== --osd-uuid 7809c935-1f4b-4a4e-82ed-f517b44da622 2026-02-18T08:58:50.259 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:50.243+0000 7fb2152e1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:50.262 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:50.247+0000 7fb2152e1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:50.263 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:50.247+0000 7fb2152e1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:50.264 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:50.247+0000 7fb2152e1a40 -1 bdev(0x55b9bcaac400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:58:50.264 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:50.247+0000 7fb2152e1a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T08:58:51.088 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T08:58:51.088 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:58:51.089 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:58:51.089 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:58:51.089 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:58:51.202 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:58:51.202 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:58:51.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:58:51.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:58:51.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:58:51.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:58:51.217 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:51.203+0000 7fc5d0068a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:51.219 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:51.203+0000 7fc5d0068a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:51.220 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:51.203+0000 7fc5d0068a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:51.316 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:58:51.316 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:58:51.316 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:58:51.317 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:58:51.317 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:58:51.317 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:51.317 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:58:51.317 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:51.318 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:51.318 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:51.426 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:51.662 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:51.647+0000 7fc5d0068a40 -1 Falling back to public interface 2026-02-18T08:58:52.380 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:52.363+0000 7fc5d0068a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:58:52.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:52.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:52.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:58:52.428 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:58:52.429 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:52.429 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:52.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:53.647 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:53.648 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:53.648 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:58:53.648 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:58:53.648 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:53.648 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:53.862 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2692471932,v1:127.0.0.1:6801/2692471932] [v2:127.0.0.1:6802/2692471932,v1:127.0.0.1:6803/2692471932] exists,up 7809c935-1f4b-4a4e-82ed-f517b44da622 2026-02-18T08:58:53.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:58:53.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:58:53.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:58:53.863 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-18T08:58:53.863 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:58:53.863 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:58:53.864 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:53.864 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:53.864 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:53.865 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-osd.0.asok 2026-02-18T08:58:53.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-18T08:58:53.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.53296/ceph-osd.0.asok config get osd_max_backfills 2026-02-18T08:58:53.924 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"1"}' 2026-02-18T08:58:53.924 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: echo '{"osd_max_backfills":"1"}' 2026-02-18T08:58:53.925 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: grep --quiet osd_max_backfills 2026-02-18T08:58:53.926 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:912: test_activate_osd_after_mark_down: kill_daemons td/ceph-helpers TERM osd 2026-02-18T08:58:53.926 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:53.926 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:53.927 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:53.927 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:53.927 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:54.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:54.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:913: test_activate_osd_after_mark_down: ceph osd down 0 2026-02-18T08:58:54.454 INFO:tasks.workunit.client.0.trial118.stderr:osd.0 is already down. 2026-02-18T08:58:54.464 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:914: test_activate_osd_after_mark_down: wait_for_osd down 0 2026-02-18T08:58:54.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-18T08:58:54.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:58:54.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:58:54.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:58:54.465 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:54.465 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:54.466 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:58:54.466 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:54.466 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-18T08:58:54.701 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/2692471932,v1:127.0.0.1:6801/2692471932] [v2:127.0.0.1:6802/2692471932,v1:127.0.0.1:6803/2692471932] exists 7809c935-1f4b-4a4e-82ed-f517b44da622 2026-02-18T08:58:54.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:58:54.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:58:54.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:58:54.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: test_activate_osd_after_mark_down: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-18T08:58:54.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-18T08:58:54.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-18T08:58:54.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:58:54.705 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:58:54.706 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:58:54.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:58:54.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:58:54.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-18T08:58:54.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-18T08:58:54.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:58:54.707 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:58:54.708 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-18T08:58:54.708 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-18T08:58:54.709 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-18T08:58:54.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-18T08:58:54.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-18T08:58:54.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-18T08:58:54.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-18T08:58:54.725 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:54.711+0000 7f7bc990ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:54.726 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:54.711+0000 7f7bc990ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:54.727 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:54.711+0000 7f7bc990ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:54.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-18T08:58:54.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:58:54.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:58:54.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:58:54.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:58:54.921 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:54.921 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:54.922 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:58:54.922 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:54.922 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:55.140 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:55.186 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:55.171+0000 7f7bc990ca40 -1 Falling back to public interface 2026-02-18T08:58:55.884 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:55.867+0000 7f7bc990ca40 -1 osd.0 5 log_to_monitors true 2026-02-18T08:58:56.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:56.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:56.142 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:58:56.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:58:56.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:56.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:56.356 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/814889432,v1:127.0.0.1:6801/814889432] [v2:127.0.0.1:6802/814889432,v1:127.0.0.1:6803/814889432] exists,up 7809c935-1f4b-4a4e-82ed-f517b44da622 2026-02-18T08:58:56.356 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:58:56.357 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:58:56.357 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:58:56.358 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-18T08:58:56.358 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:58:56.358 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:58:56.358 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:56.358 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:56.358 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:56.359 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-osd.0.asok 2026-02-18T08:58:56.359 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-18T08:58:56.359 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.53296/ceph-osd.0.asok config get osd_max_backfills 2026-02-18T08:58:56.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"20"}' 2026-02-18T08:58:56.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: test_activate_osd_after_mark_down: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-18T08:58:56.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: test_activate_osd_after_mark_down: teardown td/ceph-helpers 2026-02-18T08:58:56.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:58:56.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:58:56.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:58:56.422 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:56.423 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:56.423 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:56.423 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:56.423 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:56.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:56.530 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:58:56.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:58:56.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:58:56.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:58:56.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:58:56.533 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:58:56.534 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:56.534 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:58:56.534 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:58:56.534 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:56.535 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:58:56.536 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:58:56.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:58:56.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:58:56.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:56.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:56.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:58:56.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:58:56.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:58:56.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:58:56.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd_skip_benchmark td/ceph-helpers 2026-02-18T08:58:56.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: test_activate_osd_skip_benchmark: local dir=td/ceph-helpers 2026-02-18T08:58:56.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:927: test_activate_osd_skip_benchmark: setup td/ceph-helpers 2026-02-18T08:58:56.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:58:56.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:58:56.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:58:56.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:58:56.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:58:56.544 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:58:56.544 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:58:56.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:58:56.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:58:56.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:58:56.545 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:58:56.545 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:58:56.546 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:58:56.546 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:58:56.547 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:58:56.547 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:58:56.547 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:58:56.548 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:56.548 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:58:56.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:58:56.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:58:56.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:58:56.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:58:56.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:58:56.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:58:56.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:56.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:56.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:58:56.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:58:56.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:58:56.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:58:56.555 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:58:56.555 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:56.555 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:56.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:58:56.557 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:58:56.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:58:56.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:58:56.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:58:56.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: test_activate_osd_skip_benchmark: run_mon td/ceph-helpers a 2026-02-18T08:58:56.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:58:56.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:58:56.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:58:56.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:58:56.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:58:56.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-18T08:58:56.579 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:58:56.580 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:56.580 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:56.580 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:56.580 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:56.580 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:56.580 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:56.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:58:56.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:58:56.608 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:58:56.608 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:58:56.608 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:58:56.608 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:58:56.609 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:58:56.610 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:58:56.610 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:58:56.610 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:58:56.610 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:56.610 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:56.610 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:56.610 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:58:56.610 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:58:56.610 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:58:56.673 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:58:56.674 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:58:56.674 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:58:56.674 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:58:56.674 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:58:56.675 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:58:56.675 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:58:56.675 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:58:56.675 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:58:56.675 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:56.675 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:56.676 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:58:56.676 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:58:56.676 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:58:56.728 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:930: test_activate_osd_skip_benchmark: run_mgr td/ceph-helpers x 2026-02-18T08:58:56.729 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:58:56.729 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:58:56.729 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:58:56.729 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:58:56.729 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:58:56.729 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:58:56.838 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:58:56.838 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:56.839 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:56.839 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:56.839 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:56.839 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:56.840 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:56.840 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:58:56.841 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:58:56.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:933: test_activate_osd_skip_benchmark: run_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-18T08:58:56.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T08:58:56.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:58:56.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:58:56.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:58:56.864 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:58:56.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:58:56.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:58:56.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:58:56.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:58:56.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:58:56.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-18T08:58:56.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:58:56.866 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:58:56.867 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a8aa04db-3456-4d84-8ed4-2e28a56585c8 2026-02-18T08:58:56.867 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 a8aa04db-3456-4d84-8ed4-2e28a56585c8 2026-02-18T08:58:56.867 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a8aa04db-3456-4d84-8ed4-2e28a56585c8' 2026-02-18T08:58:56.867 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:58:56.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBQf5VpdPaPMxAAAWn29z1Lp5p9C31U6CGtHQ== 2026-02-18T08:58:56.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBQf5VpdPaPMxAAAWn29z1Lp5p9C31U6CGtHQ=="}' 2026-02-18T08:58:56.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a8aa04db-3456-4d84-8ed4-2e28a56585c8 -i td/ceph-helpers/0/new.json 2026-02-18T08:58:56.990 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:56.998 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T08:58:56.999 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true --mkfs --key AQBQf5VpdPaPMxAAAWn29z1Lp5p9C31U6CGtHQ== --osd-uuid a8aa04db-3456-4d84-8ed4-2e28a56585c8 2026-02-18T08:58:57.016 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:56.999+0000 7f4a2d5a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:57.017 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:57.003+0000 7f4a2d5a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:57.018 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:57.003+0000 7f4a2d5a0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:57.019 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:57.003+0000 7f4a2d5a0a40 -1 bdev(0x55f903a8c400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:58:57.019 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:57.003+0000 7f4a2d5a0a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T08:58:57.676 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T08:58:57.676 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:58:57.677 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:58:57.677 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:58:57.677 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:58:57.796 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:58:57.796 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:58:57.797 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-18T08:58:57.797 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:58:57.797 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:58:57.797 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:58:57.812 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:57.795+0000 7fd23a0b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:57.814 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:57.799+0000 7fd23a0b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:57.814 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:57.799+0000 7fd23a0b3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:58:57.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:58:57.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:58:57.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:58:57.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:58:57.909 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:58:57.909 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:57.909 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:58:57.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:58:57.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:57.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:58.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:58:58.242 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:58.227+0000 7fd23a0b3a40 -1 Falling back to public interface 2026-02-18T08:58:58.902 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:58:58.887+0000 7fd23a0b3a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:58:59.021 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:58:59.021 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:58:59.021 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:58:59.022 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:58:59.022 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:58:59.022 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:58:59.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:00.131 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:00.131 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:00.131 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:59:00.132 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:59:00.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:00.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:00.346 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/480516294,v1:127.0.0.1:6801/480516294] [v2:127.0.0.1:6802/480516294,v1:127.0.0.1:6803/480516294] exists,up a8aa04db-3456-4d84-8ed4-2e28a56585c8 2026-02-18T08:59:00.346 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:59:00.347 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:59:00.347 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:59:00.348 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-18T08:59:00.348 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:59:00.348 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:59:00.348 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:00.348 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:00.348 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:00.348 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-osd.0.asok 2026-02-18T08:59:00.349 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-18T08:59:00.349 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.53296/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-18T08:59:00.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: local 'max_iops_hdd_def={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-18T08:59:00.407 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-18T08:59:00.407 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:59:00.407 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:59:00.408 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:00.408 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:00.408 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:00.408 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-osd.0.asok 2026-02-18T08:59:00.409 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-18T08:59:00.409 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.53296/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-18T08:59:00.470 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: local 'max_iops_ssd_def={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-18T08:59:00.470 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:940: test_activate_osd_skip_benchmark: kill_daemons td/ceph-helpers TERM osd 2026-02-18T08:59:00.471 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:00.471 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:00.471 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:00.471 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:00.471 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:00.779 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:00.779 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:941: test_activate_osd_skip_benchmark: ceph osd down 0 2026-02-18T08:59:00.995 INFO:tasks.workunit.client.0.trial118.stderr:osd.0 is already down. 2026-02-18T08:59:01.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:942: test_activate_osd_skip_benchmark: wait_for_osd down 0 2026-02-18T08:59:01.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-18T08:59:01.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:59:01.004 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:59:01.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:59:01.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:01.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:59:01.005 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:01.006 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:01.006 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-18T08:59:01.232 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 7 last_clean_interval [0,0) [v2:127.0.0.1:6800/480516294,v1:127.0.0.1:6801/480516294] [v2:127.0.0.1:6802/480516294,v1:127.0.0.1:6803/480516294] exists a8aa04db-3456-4d84-8ed4-2e28a56585c8 2026-02-18T08:59:01.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:59:01.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:59:01.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:946: test_activate_osd_skip_benchmark: activate_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:59:01.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:59:01.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:59:01.238 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:59:01.238 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:59:01.238 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:59:01.238 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:59:01.238 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:59:01.238 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-18T08:59:01.238 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-18T08:59:01.238 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:59:01.238 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:59:01.239 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-18T08:59:01.240 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-18T08:59:01.240 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-18T08:59:01.240 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-18T08:59:01.240 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-18T08:59:01.240 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-18T08:59:01.240 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-18T08:59:01.254 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:01.239+0000 7fd84403fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:01.256 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:01.239+0000 7fd84403fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:01.257 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:01.243+0000 7fd84403fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:01.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-18T08:59:01.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:59:01.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:59:01.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:59:01.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:59:01.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:01.447 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:59:01.447 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:01.447 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:01.448 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:01.658 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:01.643+0000 7fd84403fa40 -1 Falling back to public interface 2026-02-18T08:59:01.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:02.297 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:02.283+0000 7fd84403fa40 -1 osd.0 6 log_to_monitors true 2026-02-18T08:59:02.669 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:02.669 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:02.669 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:59:02.669 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:59:02.669 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:02.669 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:02.883 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 9 up_thru 0 down_at 7 last_clean_interval [5,6) [v2:127.0.0.1:6800/2838168705,v1:127.0.0.1:6801/2838168705] [v2:127.0.0.1:6802/2838168705,v1:127.0.0.1:6803/2838168705] exists,up a8aa04db-3456-4d84-8ed4-2e28a56585c8 2026-02-18T08:59:02.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:59:02.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:59:02.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:59:02.886 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-18T08:59:02.886 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:59:02.886 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:59:02.886 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:02.886 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:02.886 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:02.886 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-osd.0.asok 2026-02-18T08:59:02.886 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-18T08:59:02.887 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.53296/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-18T08:59:02.945 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: local 'max_iops_hdd_after_boot={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-18T08:59:02.946 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-18T08:59:02.946 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:59:02.946 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:59:02.946 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:02.947 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:02.947 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:02.947 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-osd.0.asok 2026-02-18T08:59:02.947 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-18T08:59:02.947 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.53296/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-18T08:59:03.007 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: local 'max_iops_ssd_after_boot={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-18T08:59:03.007 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:953: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' = '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-18T08:59:03.007 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:954: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' = '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-18T08:59:03.007 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:956: test_activate_osd_skip_benchmark: teardown td/ceph-helpers 2026-02-18T08:59:03.007 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:03.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:03.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:03.008 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:03.008 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:03.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:03.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:03.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:03.115 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:03.115 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:03.116 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:03.116 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:03.117 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:03.117 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:03.118 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:03.118 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:03.119 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:03.119 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:03.119 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:03.120 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:03.121 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:03.121 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:03.124 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:03.125 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:03.125 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:03.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:03.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:03.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:03.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:03.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_destroy_osd td/ceph-helpers 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:787: test_destroy_osd: local dir=td/ceph-helpers 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:789: test_destroy_osd: setup td/ceph-helpers 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:03.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:03.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:03.129 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:03.130 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:03.130 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:03.131 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:03.131 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:03.132 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:03.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:03.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:03.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:03.134 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:03.134 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:03.135 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:03.135 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:03.136 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:03.137 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:03.137 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:03.137 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:03.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:03.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:03.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:03.139 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:03.139 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:03.139 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:03.139 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:03.141 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:03.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:03.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:03.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:03.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:790: test_destroy_osd: run_mon td/ceph-helpers a 2026-02-18T08:59:03.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:59:03.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:59:03.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:59:03.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:59:03.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:59:03.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-18T08:59:03.163 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:59:03.163 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:03.163 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:03.164 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:03.164 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:03.164 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:03.164 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:03.164 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:59:03.192 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:59:03.193 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:59:03.193 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:03.193 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:03.193 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:59:03.193 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:59:03.195 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:03.195 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:03.195 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:03.195 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:03.195 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:03.195 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:03.195 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:03.195 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:03.196 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:59:03.250 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:59:03.250 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:03.250 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:03.250 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:59:03.250 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:59:03.251 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:03.251 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:03.251 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:03.251 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:03.252 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:03.252 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:03.252 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:03.252 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:03.252 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:59:03.306 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:791: test_destroy_osd: run_mgr td/ceph-helpers x 2026-02-18T08:59:03.306 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:59:03.306 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:59:03.307 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:59:03.307 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:59:03.307 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:59:03.307 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:59:03.421 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:59:03.421 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:03.422 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:03.422 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:03.422 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:03.422 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:03.422 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:03.422 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:03.423 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:03.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:792: test_destroy_osd: run_osd td/ceph-helpers 0 2026-02-18T08:59:03.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T08:59:03.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:59:03.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:59:03.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:59:03.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:59:03.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:59:03.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:03.444 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:59:03.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:59:03.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:59:03.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:59:03.446 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:59:03.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bb7d6638-ce3f-4213-8b9b-3fb245ae8b43 2026-02-18T08:59:03.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 bb7d6638-ce3f-4213-8b9b-3fb245ae8b43' 2026-02-18T08:59:03.447 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 bb7d6638-ce3f-4213-8b9b-3fb245ae8b43 2026-02-18T08:59:03.447 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:59:03.455 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBXf5VpOkd+GhAAo0lCqksaHu4/H4YXlTqxjA== 2026-02-18T08:59:03.455 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBXf5VpOkd+GhAAo0lCqksaHu4/H4YXlTqxjA=="}' 2026-02-18T08:59:03.455 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bb7d6638-ce3f-4213-8b9b-3fb245ae8b43 -i td/ceph-helpers/0/new.json 2026-02-18T08:59:03.565 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:03.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T08:59:03.574 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBXf5VpOkd+GhAAo0lCqksaHu4/H4YXlTqxjA== --osd-uuid bb7d6638-ce3f-4213-8b9b-3fb245ae8b43 2026-02-18T08:59:03.590 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:03.575+0000 7fc5d0467a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:03.591 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:03.575+0000 7fc5d0467a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:03.592 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:03.575+0000 7fc5d0467a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:03.593 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:03.579+0000 7fc5d0467a40 -1 bdev(0x5591bd292400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:59:03.593 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:03.579+0000 7fc5d0467a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T08:59:04.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T08:59:04.352 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:59:04.353 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:59:04.354 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:59:04.354 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:59:04.478 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:59:04.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:59:04.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:59:04.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:59:04.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:59:04.479 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:59:04.495 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:04.479+0000 7f14612b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:04.496 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:04.479+0000 7f14612b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:04.497 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:04.483+0000 7f14612b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:04.592 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:59:04.593 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:59:04.593 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:59:04.593 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:59:04.593 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:59:04.593 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:04.593 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:04.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:59:04.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:04.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:04.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:04.983 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:04.967+0000 7f14612b5a40 -1 Falling back to public interface 2026-02-18T08:59:05.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:05.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:05.704 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:59:05.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:59:05.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:05.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:05.814 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:05.962 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:05.947+0000 7f14612b5a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:59:06.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:06.816 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:06.816 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:59:06.817 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:59:06.817 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:06.817 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:07.033 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1580893461,v1:127.0.0.1:6801/1580893461] [v2:127.0.0.1:6802/1580893461,v1:127.0.0.1:6803/1580893461] exists,up bb7d6638-ce3f-4213-8b9b-3fb245ae8b43 2026-02-18T08:59:07.034 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:59:07.034 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:59:07.034 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:59:07.034 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:793: test_destroy_osd: destroy_osd td/ceph-helpers 0 2026-02-18T08:59:07.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:775: destroy_osd: local dir=td/ceph-helpers 2026-02-18T08:59:07.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:776: destroy_osd: local id=0 2026-02-18T08:59:07.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:778: destroy_osd: ceph osd out osd.0 2026-02-18T08:59:07.280 INFO:tasks.workunit.client.0.trial118.stderr:osd.0 is already out. 2026-02-18T08:59:07.290 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:779: destroy_osd: kill_daemons td/ceph-helpers TERM osd.0 2026-02-18T08:59:07.291 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:07.291 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:07.291 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:07.291 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:07.291 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:07.599 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:07.599 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:780: destroy_osd: ceph osd down osd.0 2026-02-18T08:59:07.814 INFO:tasks.workunit.client.0.trial118.stderr:osd.0 is already down. 2026-02-18T08:59:07.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:781: destroy_osd: ceph osd purge osd.0 --yes-i-really-mean-it 2026-02-18T08:59:08.027 INFO:tasks.workunit.client.0.trial118.stderr:osd.0 does not exist 2026-02-18T08:59:08.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:782: destroy_osd: teardown td/ceph-helpers/0 2026-02-18T08:59:08.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers/0 2026-02-18T08:59:08.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:08.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers/0 KILL 2026-02-18T08:59:08.038 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:08.038 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:08.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:08.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:08.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:08.040 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:08.040 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:08.041 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:08.041 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:08.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:08.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:08.043 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:08.044 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:08.044 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:08.044 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:08.044 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:08.045 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:08.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:08.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers/0 2026-02-18T08:59:08.049 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:08.049 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.049 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.049 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:08.050 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:08.050 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:08.050 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:783: destroy_osd: rm -fr td/ceph-helpers/0 2026-02-18T08:59:08.051 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: ceph osd dump 2026-02-18T08:59:08.052 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: grep 'osd. ' 2026-02-18T08:59:08.275 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:795: test_destroy_osd: teardown td/ceph-helpers 2026-02-18T08:59:08.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:08.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:08.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:08.276 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:08.276 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:08.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:08.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:08.276 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:08.383 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:08.384 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:08.384 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:08.385 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:08.386 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:08.386 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:08.386 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:08.387 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:08.387 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:08.388 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:08.388 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:08.389 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:08.390 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:08.390 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:08.393 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:08.393 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.393 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.393 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:08.395 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_display_logs td/ceph-helpers 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2158: test_display_logs: local dir=td/ceph-helpers 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2160: test_display_logs: setup td/ceph-helpers 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:08.396 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:08.397 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:08.397 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:08.397 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:08.397 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:08.398 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:08.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:08.399 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:08.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:08.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:08.400 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:08.400 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:08.401 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:08.401 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:08.401 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:08.402 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:08.403 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:08.403 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:08.404 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:08.404 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.404 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:08.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:08.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:08.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:08.407 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:08.407 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.407 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.407 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:08.409 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:08.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:08.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:08.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:08.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2161: test_display_logs: run_mon td/ceph-helpers a 2026-02-18T08:59:08.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:59:08.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:59:08.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:59:08.410 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:59:08.411 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:59:08.411 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-18T08:59:08.432 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:59:08.432 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:08.432 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:08.432 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:08.433 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.433 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.433 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:08.433 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:59:08.462 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:59:08.463 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:59:08.464 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:08.464 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:08.464 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:59:08.464 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:59:08.465 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:08.465 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:08.465 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:08.465 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:08.465 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.466 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.466 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:08.466 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:08.466 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:59:08.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:59:08.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:08.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:08.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:59:08.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:59:08.522 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:08.523 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:08.523 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:08.523 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:08.523 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.523 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.523 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:08.523 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:08.523 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:59:08.577 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2162: test_display_logs: kill_daemons td/ceph-helpers 2026-02-18T08:59:08.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:08.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:08.578 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:08.578 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:08.578 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:08.684 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:08.684 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2163: test_display_logs: display_logs td/ceph-helpers 2026-02-18T08:59:08.684 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2148: display_logs: local dir=td/ceph-helpers 2026-02-18T08:59:08.684 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: display_logs: find td/ceph-helpers -maxdepth 1 -name '*.log' 2026-02-18T08:59:08.684 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-18T08:59:08.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2152: display_logs: echo '======================= td/ceph-helpers/mon.a.log' 2026-02-18T08:59:08.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2153: display_logs: cat td/ceph-helpers/mon.a.log 2026-02-18T08:59:08.686 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-18T08:59:08.687 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2164: test_display_logs: grep --quiet mon.a.log td/ceph-helpers/log.out 2026-02-18T08:59:08.688 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2165: test_display_logs: teardown td/ceph-helpers 2026-02-18T08:59:08.688 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:08.688 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:08.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:08.689 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:08.689 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:08.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:08.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:08.689 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:08.690 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:08.691 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:08.691 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:08.692 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:08.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:08.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:08.694 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:08.695 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:08.695 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:08.695 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:08.695 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:08.696 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:08.697 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:08.697 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:08.699 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:08.699 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.699 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.699 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:08.701 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:08.701 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:08.701 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:08.701 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_erasure_code_plugin_exists td/ceph-helpers 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: test_erasure_code_plugin_exists: local dir=td/ceph-helpers 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: test_erasure_code_plugin_exists: setup td/ceph-helpers 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:08.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:08.703 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:08.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:08.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:08.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:08.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:08.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:08.705 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:08.706 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:08.706 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:08.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:08.707 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:08.708 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:08.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:08.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:08.710 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:08.710 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.711 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.711 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:08.712 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:08.712 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:08.712 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:08.713 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:08.713 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.713 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.713 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:08.715 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:08.716 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:08.716 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:08.716 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:08.716 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: test_erasure_code_plugin_exists: run_mon td/ceph-helpers a 2026-02-18T08:59:08.716 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:59:08.716 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:59:08.716 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:59:08.717 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:59:08.717 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:59:08.717 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-18T08:59:08.737 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:59:08.737 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:08.737 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:08.737 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:08.737 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.737 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:08.738 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:59:08.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:59:08.770 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:59:08.770 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:08.770 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:08.770 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:59:08.770 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:59:08.770 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:08.770 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:08.771 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:08.771 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:08.771 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.771 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.771 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:08.771 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:08.772 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:59:08.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:59:08.836 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:08.836 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:08.836 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:59:08.836 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:59:08.836 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:08.836 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:08.836 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:08.836 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:08.836 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:08.836 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:08.837 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:08.837 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:08.837 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:59:08.892 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: test_erasure_code_plugin_exists: run_mgr td/ceph-helpers x 2026-02-18T08:59:08.892 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:59:08.892 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:59:08.892 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:59:08.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:59:08.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:59:08.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:59:09.006 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:59:09.007 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:09.007 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:09.007 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:09.007 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.007 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.007 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:09.008 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:09.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:09.025 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2134: test_erasure_code_plugin_exists: erasure_code_plugin_exists jerasure 2026-02-18T08:59:09.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=jerasure 2026-02-18T08:59:09.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-18T08:59:09.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-18T08:59:09.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-18T08:59:09.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-18T08:59:09.026 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-18T08:59:09.027 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='jerasure.*No such file' 2026-02-18T08:59:09.027 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=jerasure 2026-02-18T08:59:09.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-18T08:59:09.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-18T08:59:09.154 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-18T08:59:09.154 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-18T08:59:09.300 INFO:tasks.workunit.client.0.trial118.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-18T08:59:09.308 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-18T08:59:09.309 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2135: test_erasure_code_plugin_exists: erasure_code_plugin_exists FAKE 2026-02-18T08:59:09.309 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=FAKE 2026-02-18T08:59:09.309 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-18T08:59:09.309 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-18T08:59:09.309 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-18T08:59:09.309 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-18T08:59:09.309 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-18T08:59:09.310 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='FAKE.*No such file' 2026-02-18T08:59:09.310 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=FAKE 2026-02-18T08:59:09.403 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s='Error EIO: load dlopen(/usr/lib/ceph/erasure-code/libec_FAKE.so): /usr/lib/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory' 2026-02-18T08:59:09.403 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=5 2026-02-18T08:59:09.404 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 5 -eq 0 ']' 2026-02-18T08:59:09.404 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: echo Error EIO: load 'dlopen(/usr/lib/ceph/erasure-code/libec_FAKE.so):' /usr/lib/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory 2026-02-18T08:59:09.404 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: grep --quiet 'FAKE.*No such file' 2026-02-18T08:59:09.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2119: erasure_code_plugin_exists: status=1 2026-02-18T08:59:09.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 1 2026-02-18T08:59:09.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2136: test_erasure_code_plugin_exists: teardown td/ceph-helpers 2026-02-18T08:59:09.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:09.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:09.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:09.406 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:09.406 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:09.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:09.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:09.406 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:09.512 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:09.512 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:09.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:09.513 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:09.515 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:09.515 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:09.515 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:09.516 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.516 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:09.517 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:09.517 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.518 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:09.519 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:09.519 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:09.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:09.521 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.522 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:09.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:09.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:09.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:09.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_expect_failure td/ceph-helpers 2026-02-18T08:59:09.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: test_expect_failure: local dir=td/ceph-helpers 2026-02-18T08:59:09.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2039: test_expect_failure: setup td/ceph-helpers 2026-02-18T08:59:09.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:09.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:09.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:09.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:09.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:09.525 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:09.525 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:09.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:09.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:09.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:09.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:09.526 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:09.526 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:09.527 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:09.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:09.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:09.528 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:09.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:09.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:09.529 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:09.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:09.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:09.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:09.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:09.534 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:09.534 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:09.534 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:09.535 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:09.535 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.535 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.535 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:09.536 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:09.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:09.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:09.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:09.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2040: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 1' 2026-02-18T08:59:09.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-18T08:59:09.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-18T08:59:09.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-18T08:59:09.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-18T08:59:09.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-18T08:59:09.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 1' 2026-02-18T08:59:09.539 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-18T08:59:09.539 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-18T08:59:09.539 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-18T08:59:09.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-18T08:59:09.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2042: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 0' 2026-02-18T08:59:09.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-18T08:59:09.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-18T08:59:09.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-18T08:59:09.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-18T08:59:09.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-18T08:59:09.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 0' 2026-02-18T08:59:09.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2023: expect_failure: success=true 2026-02-18T08:59:09.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: true 2026-02-18T08:59:09.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-18T08:59:09.542 INFO:tasks.workunit.client.0.trial118.stderr:FAIL 2026-02-18T08:59:09.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-18T08:59:09.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2043: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-18T08:59:09.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2045: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo UNEXPECTED ; exit 1' 2026-02-18T08:59:09.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-18T08:59:09.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-18T08:59:09.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-18T08:59:09.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-18T08:59:09.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-18T08:59:09.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo UNEXPECTED ; exit 1' 2026-02-18T08:59:09.545 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-18T08:59:09.545 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-18T08:59:09.545 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-18T08:59:09.546 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-18T08:59:09.547 INFO:tasks.workunit.client.0.trial118.stderr:UNEXPECTED 2026-02-18T08:59:09.547 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-18T08:59:09.547 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2046: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-18T08:59:09.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2047: test_expect_failure: teardown td/ceph-helpers 2026-02-18T08:59:09.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:09.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:09.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:09.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:09.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:09.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:09.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:09.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:09.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:09.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:09.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:09.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:09.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:09.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:09.554 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:09.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:09.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:09.556 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.557 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:09.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:09.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:09.559 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:09.560 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.560 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:09.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:09.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:09.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:09.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_file_with_random_data td/ceph-helpers 2026-02-18T08:59:09.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2350: test_file_with_random_data: local dir=td/ceph-helpers 2026-02-18T08:59:09.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2352: test_file_with_random_data: setup td/ceph-helpers 2026-02-18T08:59:09.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:09.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:09.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:09.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:09.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:09.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:09.563 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:09.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:09.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:09.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:09.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:09.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:09.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:09.565 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:09.566 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:09.566 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:09.567 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:09.568 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.568 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:09.568 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:09.568 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.569 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:09.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:09.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:09.571 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:09.572 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.572 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:09.573 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:09.573 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:09.573 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:09.574 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:09.574 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.574 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.574 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:09.576 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:09.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:09.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:09.577 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:09.577 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2353: test_file_with_random_data: file_with_random_data 4000 2026-02-18T08:59:09.577 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2343: file_with_random_data: local size_bytes=4000 2026-02-18T08:59:09.577 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: mktemp 2026-02-18T08:59:09.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local file=/tmp/tmp.AfGb3mpBjo 2026-02-18T08:59:09.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.AfGb3mpBjo bs=4000 count=1 2026-02-18T08:59:09.579 INFO:tasks.workunit.client.0.trial118.stderr:1+0 records in 2026-02-18T08:59:09.579 INFO:tasks.workunit.client.0.trial118.stderr:1+0 records out 2026-02-18T08:59:09.579 INFO:tasks.workunit.client.0.trial118.stderr:4000 bytes (4.0 kB, 3.9 KiB) copied, 0.000128658 s, 31.1 MB/s 2026-02-18T08:59:09.579 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: printf %s /tmp/tmp.AfGb3mpBjo 2026-02-18T08:59:09.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2353: test_file_with_random_data: local file=/tmp/tmp.AfGb3mpBjo 2026-02-18T08:59:09.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2354: test_file_with_random_data: test -f /tmp/tmp.AfGb3mpBjo 2026-02-18T08:59:09.580 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2355: test_file_with_random_data: stat -c %s /tmp/tmp.AfGb3mpBjo 2026-02-18T08:59:09.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2355: test_file_with_random_data: test 4000 = 4000 2026-02-18T08:59:09.581 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2356: test_file_with_random_data: rm /tmp/tmp.AfGb3mpBjo 2026-02-18T08:59:09.582 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2357: test_file_with_random_data: teardown td/ceph-helpers 2026-02-18T08:59:09.582 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:09.582 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:09.582 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:09.582 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:09.583 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:09.583 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:09.583 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:09.583 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:09.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:09.585 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:09.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:09.586 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:09.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:09.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:09.587 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:09.588 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.588 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:09.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:09.589 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:09.590 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:09.590 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:09.592 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:09.592 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.592 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.592 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:09.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:09.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:09.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:09.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_flush_pg_stats td/ceph-helpers 2026-02-18T08:59:09.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2280: test_flush_pg_stats: local dir=td/ceph-helpers 2026-02-18T08:59:09.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: test_flush_pg_stats: setup td/ceph-helpers 2026-02-18T08:59:09.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:09.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:09.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:09.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:09.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:09.595 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:09.595 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:09.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:09.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:09.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:09.596 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:09.596 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:09.597 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:09.597 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:09.598 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:09.598 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:09.599 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:09.600 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.600 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:09.600 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:09.601 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:09.601 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:09.602 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:09.602 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:09.603 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:09.604 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.604 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.604 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:09.605 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:09.605 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:09.605 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:09.606 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:09.606 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.607 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:09.608 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:09.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:09.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:09.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:09.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2283: test_flush_pg_stats: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:09.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:59:09.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:59:09.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:59:09.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:59:09.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:59:09.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:09.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:59:09.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:09.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:09.630 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:09.630 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.630 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:09.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:09.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:59:09.669 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:59:09.669 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:09.669 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:09.669 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:59:09.670 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:59:09.670 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:09.670 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:09.671 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:09.671 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:09.671 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.671 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.671 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:09.671 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:09.671 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:59:09.726 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:59:09.726 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:09.726 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:09.726 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:59:09.727 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:59:09.727 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:09.727 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:09.727 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:09.728 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:09.728 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.728 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.728 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:09.728 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:09.728 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:59:09.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: test_flush_pg_stats: run_mgr td/ceph-helpers x 2026-02-18T08:59:09.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:59:09.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:59:09.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:59:09.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:59:09.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:59:09.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:59:09.893 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:59:09.894 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:09.894 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:09.894 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:09.894 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.894 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.894 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:09.894 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:09.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:09.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: test_flush_pg_stats: run_osd td/ceph-helpers 0 2026-02-18T08:59:09.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T08:59:09.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:59:09.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:59:09.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:59:09.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:59:09.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:59:09.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:59:09.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:59:09.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:09.917 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:59:09.918 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:59:09.919 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:59:09.919 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b267de7c-bd91-4beb-b49f-1b8abb86a991 2026-02-18T08:59:09.920 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 b267de7c-bd91-4beb-b49f-1b8abb86a991 2026-02-18T08:59:09.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b267de7c-bd91-4beb-b49f-1b8abb86a991' 2026-02-18T08:59:09.920 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:59:09.928 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBdf5Vp5u+sNhAADKABXM98XPePzk9yC335pQ== 2026-02-18T08:59:09.928 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBdf5Vp5u+sNhAADKABXM98XPePzk9yC335pQ=="}' 2026-02-18T08:59:09.928 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b267de7c-bd91-4beb-b49f-1b8abb86a991 -i td/ceph-helpers/0/new.json 2026-02-18T08:59:10.041 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:10.049 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T08:59:10.050 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBdf5Vp5u+sNhAADKABXM98XPePzk9yC335pQ== --osd-uuid b267de7c-bd91-4beb-b49f-1b8abb86a991 2026-02-18T08:59:10.066 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:10.051+0000 7f3124f05a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:10.068 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:10.051+0000 7f3124f05a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:10.069 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:10.055+0000 7f3124f05a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:10.069 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:10.055+0000 7f3124f05a40 -1 bdev(0x557dc8132400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:59:10.070 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:10.055+0000 7f3124f05a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T08:59:10.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T08:59:10.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:59:10.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:59:10.826 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:59:10.826 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:59:10.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:59:10.949 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:59:10.950 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:59:10.950 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:59:10.950 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:59:10.950 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:59:10.974 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:10.959+0000 7ff4a8283a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:10.976 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:10.959+0000 7ff4a8283a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:10.978 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:10.963+0000 7ff4a8283a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:11.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:59:11.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:59:11.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:59:11.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:59:11.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:59:11.062 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:11.062 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:11.062 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:59:11.062 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:11.062 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:11.172 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:11.507 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:11.491+0000 7ff4a8283a40 -1 Falling back to public interface 2026-02-18T08:59:12.173 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:12.173 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:12.174 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:59:12.174 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:59:12.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:12.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:12.284 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:12.402 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:12.387+0000 7ff4a8283a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:59:13.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:13.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:13.286 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:59:13.287 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:59:13.287 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:13.287 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:13.501 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1481808244,v1:127.0.0.1:6801/1481808244] [v2:127.0.0.1:6802/1481808244,v1:127.0.0.1:6803/1481808244] exists,up b267de7c-bd91-4beb-b49f-1b8abb86a991 2026-02-18T08:59:13.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:59:13.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:59:13.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:59:13.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: test_flush_pg_stats: create_rbd_pool 2026-02-18T08:59:13.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T08:59:13.710 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T08:59:13.719 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T08:59:13.719 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T08:59:14.002 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T08:59:14.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T08:59:15.013 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T08:59:15.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: test_flush_pg_stats: rados -p rbd put obj /etc/group 2026-02-18T08:59:15.297 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: test_flush_pg_stats: flush_pg_stats 2026-02-18T08:59:15.297 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:59:15.297 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:59:15.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T08:59:15.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:59:15.524 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:59:15.524 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:59:15.603 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-18T08:59:15.603 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-18T08:59:15.604 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-18T08:59:15.604 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:59:15.604 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-18T08:59:15.604 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:59:15.605 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:59:15.605 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-18T08:59:15.605 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:59:15.606 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-18T08:59:15.607 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836483 2026-02-18T08:59:15.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-18T08:59:15.607 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:59:15.832 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-18T08:59:15.832 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:59:16.833 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:59:16.834 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:59:17.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-18T08:59:17.047 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: test_flush_pg_stats: local 'jq_filter=.pools | .[] | select(.name == "rbd") | .stats' 2026-02-18T08:59:17.047 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: ceph df detail --format=json 2026-02-18T08:59:17.047 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored' 2026-02-18T08:59:17.337 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: stored=8348 2026-02-18T08:59:17.337 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: ceph df detail --format=json 2026-02-18T08:59:17.337 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored_raw' 2026-02-18T08:59:17.629 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: stored_raw=8348 2026-02-18T08:59:17.630 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2292: test_flush_pg_stats: test 8348 -gt 0 2026-02-18T08:59:17.630 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2293: test_flush_pg_stats: test 8348 == 8348 2026-02-18T08:59:17.630 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: test_flush_pg_stats: teardown td/ceph-helpers 2026-02-18T08:59:17.630 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:17.630 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:17.630 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:17.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:17.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:17.630 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:17.630 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:17.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:17.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:17.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:17.738 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:17.739 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:17.740 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:17.740 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:17.741 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:17.742 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:17.742 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:17.742 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:17.742 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:17.743 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:17.744 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:17.744 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:17.749 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:17.749 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:17.749 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:17.749 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_config td/ceph-helpers 2026-02-18T08:59:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: test_get_config: local dir=td/ceph-helpers 2026-02-18T08:59:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: test_get_config: setup td/ceph-helpers 2026-02-18T08:59:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:17.753 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:17.753 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:17.753 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:17.753 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:17.753 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:17.753 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:17.754 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:17.755 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:17.755 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:17.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:17.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:17.756 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:17.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:17.758 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:17.758 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:17.758 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:17.759 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:17.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:17.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:17.761 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:17.762 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:17.762 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:17.762 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:17.764 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:17.764 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:17.765 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:17.765 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:17.766 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:17.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:17.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:17.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:17.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1131: test_get_config: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:17.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:59:17.768 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:59:17.768 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:59:17.768 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:59:17.768 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:59:17.768 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:17.788 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:59:17.789 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:17.789 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:17.789 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:17.789 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:17.789 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:17.789 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:17.790 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:17.815 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:59:17.816 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:59:17.816 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:17.816 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:17.816 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:59:17.818 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:59:17.818 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:17.818 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:17.818 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:17.818 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:17.818 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:17.818 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:17.818 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:17.818 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:17.818 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:59:17.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:59:17.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:17.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:17.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:59:17.873 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:59:17.874 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:17.874 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:17.874 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:17.874 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:17.874 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:17.874 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:17.874 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:17.874 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:17.874 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:59:17.927 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: get_config mon a osd_pool_default_size 2026-02-18T08:59:17.927 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:17.928 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:17.928 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_pool_default_size 2026-02-18T08:59:17.928 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_pool_default_size 2026-02-18T08:59:17.929 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:17.929 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:17.929 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:17.929 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:17.929 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:17.929 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:17.929 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:17.930 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:17.930 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get osd_pool_default_size 2026-02-18T08:59:17.986 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: test 1 = 1 2026-02-18T08:59:17.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1133: test_get_config: run_mgr td/ceph-helpers x 2026-02-18T08:59:17.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:59:17.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:59:17.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:59:17.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:59:17.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:59:17.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:59:18.105 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:59:18.105 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:18.105 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:18.105 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:18.105 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:18.106 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:18.106 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:18.106 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:18.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:18.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1134: test_get_config: run_osd td/ceph-helpers 0 --osd_max_scrubs=3 2026-02-18T08:59:18.125 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T08:59:18.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:59:18.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:59:18.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:59:18.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:59:18.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:59:18.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:59:18.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:59:18.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:59:18.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:59:18.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:59:18.126 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:59:18.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:59:18.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:59:18.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:59:18.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:59:18.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:59:18.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:59:18.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:59:18.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:59:18.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:59:18.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_max_scrubs=3 2026-02-18T08:59:18.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:59:18.129 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:59:18.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=78171656-ccb0-486d-8caa-c87c5d8aa306 2026-02-18T08:59:18.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 78171656-ccb0-486d-8caa-c87c5d8aa306' 2026-02-18T08:59:18.129 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 78171656-ccb0-486d-8caa-c87c5d8aa306 2026-02-18T08:59:18.130 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:59:18.139 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBmf5VpODSfBxAAoFFMc8hs01bvCulNq7TTVQ== 2026-02-18T08:59:18.139 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBmf5VpODSfBxAAoFFMc8hs01bvCulNq7TTVQ=="}' 2026-02-18T08:59:18.139 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 78171656-ccb0-486d-8caa-c87c5d8aa306 -i td/ceph-helpers/0/new.json 2026-02-18T08:59:18.251 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:18.259 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T08:59:18.260 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 --mkfs --key AQBmf5VpODSfBxAAoFFMc8hs01bvCulNq7TTVQ== --osd-uuid 78171656-ccb0-486d-8caa-c87c5d8aa306 2026-02-18T08:59:18.276 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:18.263+0000 7f5600919a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:18.278 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:18.267+0000 7f5600919a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:18.279 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:18.267+0000 7f5600919a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:18.280 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:18.267+0000 7f5600919a40 -1 bdev(0x556bf174e400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:59:18.280 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:18.267+0000 7f5600919a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T08:59:19.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T08:59:19.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:59:19.037 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:59:19.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:59:19.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:59:19.157 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:59:19.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:59:19.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 2026-02-18T08:59:19.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:59:19.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:59:19.158 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:59:19.183 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:19.171+0000 7f524f1eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:19.185 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:19.175+0000 7f524f1eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:19.187 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:19.175+0000 7f524f1eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:19.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:59:19.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:59:19.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:59:19.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:59:19.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:59:19.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:19.268 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:19.268 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:59:19.268 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:19.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:19.373 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:19.739 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:19.727+0000 7f524f1eba40 -1 Falling back to public interface 2026-02-18T08:59:20.374 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:20.375 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:20.375 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:59:20.376 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:59:20.376 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:20.376 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:20.485 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:20.658 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:20.647+0000 7f524f1eba40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:59:21.487 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:21.487 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:21.487 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:59:21.488 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:59:21.488 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:21.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:21.702 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4256360779,v1:127.0.0.1:6801/4256360779] [v2:127.0.0.1:6802/4256360779,v1:127.0.0.1:6803/4256360779] exists,up 78171656-ccb0-486d-8caa-c87c5d8aa306 2026-02-18T08:59:21.703 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:59:21.703 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:59:21.703 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:59:21.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: get_config osd 0 osd_max_scrubs 2026-02-18T08:59:21.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-18T08:59:21.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-18T08:59:21.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_max_scrubs 2026-02-18T08:59:21.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_max_scrubs 2026-02-18T08:59:21.704 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-18T08:59:21.704 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T08:59:21.704 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T08:59:21.705 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:21.705 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:21.705 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:21.705 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-osd.0.asok 2026-02-18T08:59:21.706 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:21.706 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-osd.0.asok config get osd_max_scrubs 2026-02-18T08:59:21.766 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: test 3 = 3 2026-02-18T08:59:21.766 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1136: test_get_config: teardown td/ceph-helpers 2026-02-18T08:59:21.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:21.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:21.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:21.767 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:21.767 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:21.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:21.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:21.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:21.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:21.875 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:21.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:21.876 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:21.877 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:21.877 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:21.877 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:21.878 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:21.878 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:21.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:21.879 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:21.880 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:21.881 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:21.881 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:21.884 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:21.885 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:21.885 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:21.885 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:21.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:21.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:21.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:21.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_is_making_recovery_progress td/ceph-helpers 2026-02-18T08:59:21.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: test_get_is_making_recovery_progress: local dir=td/ceph-helpers 2026-02-18T08:59:21.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: test_get_is_making_recovery_progress: setup td/ceph-helpers 2026-02-18T08:59:21.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:21.887 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:21.888 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:21.888 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:21.888 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:21.888 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:21.888 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:21.888 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:21.888 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:21.888 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:21.889 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:21.889 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:21.889 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:21.890 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:21.891 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:21.891 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:21.891 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:21.892 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:21.892 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:21.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:21.893 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:21.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:21.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:21.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:21.896 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:21.896 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:21.896 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:21.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:21.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:21.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:21.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:21.898 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:21.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:21.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:21.899 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:21.900 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:21.901 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:21.901 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:21.902 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:21.902 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1338: test_get_is_making_recovery_progress: run_mon td/ceph-helpers a 2026-02-18T08:59:21.902 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:59:21.902 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:59:21.902 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:59:21.902 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:59:21.902 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:59:21.902 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-18T08:59:21.921 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:59:21.922 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:21.922 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:21.922 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:21.922 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:21.922 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:21.922 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:21.923 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T08:59:21.952 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:59:21.953 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:59:21.953 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:21.953 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:21.953 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:59:21.954 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:59:21.955 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:21.955 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:21.955 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:21.955 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:21.955 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:21.955 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:21.955 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:21.955 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:21.955 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:59:22.009 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:59:22.009 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:22.009 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:22.009 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:59:22.010 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:59:22.011 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:22.011 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:22.011 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:22.011 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:22.011 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:22.011 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:22.011 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:22.011 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:22.011 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:59:22.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: test_get_is_making_recovery_progress: run_mgr td/ceph-helpers x 2026-02-18T08:59:22.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:59:22.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:59:22.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:59:22.064 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:59:22.065 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:59:22.065 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:59:22.173 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:59:22.173 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:22.173 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:22.173 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:22.173 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:22.174 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:22.174 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:22.174 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:22.175 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:22.194 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: test_get_is_making_recovery_progress: get_is_making_recovery_progress 2026-02-18T08:59:22.195 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-18T08:59:22.195 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-18T08:59:22.195 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-18T08:59:22.195 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-18T08:59:22.195 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-18T08:59:22.195 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-18T08:59:22.310 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-18T08:59:22.310 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-18T08:59:22.311 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1341: test_get_is_making_recovery_progress: teardown td/ceph-helpers 2026-02-18T08:59:22.311 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:22.311 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:22.311 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:22.311 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:22.311 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:22.311 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:22.311 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:22.311 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:22.418 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:22.418 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:22.419 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:22.419 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:22.421 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:22.421 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:22.421 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:22.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:22.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:22.422 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:22.423 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:22.423 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:22.425 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:22.425 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:22.427 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:22.427 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:22.427 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:22.427 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:22.429 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_last_scrub_stamp td/ceph-helpers 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1524: test_get_last_scrub_stamp: local dir=td/ceph-helpers 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: test_get_last_scrub_stamp: setup td/ceph-helpers 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:22.430 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:22.431 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:22.431 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:22.431 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:22.431 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:22.432 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:22.432 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:22.433 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:22.434 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:22.434 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:22.434 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:22.435 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:22.435 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:22.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:22.436 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:22.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:22.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:22.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:22.439 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:22.439 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:22.439 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:22.439 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:22.440 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:22.441 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:22.441 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:22.442 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:22.442 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:22.442 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:22.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:22.444 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:22.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:22.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:22.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:22.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: test_get_last_scrub_stamp: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:22.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:59:22.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:59:22.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:59:22.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:59:22.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:59:22.446 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:22.466 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:59:22.466 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:22.466 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:22.466 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:22.466 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:22.466 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:22.467 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:22.467 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:22.495 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:59:22.496 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:59:22.496 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:22.496 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:22.496 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:59:22.497 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:59:22.497 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:22.498 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:22.498 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:22.498 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:22.498 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:22.498 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:22.498 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:22.498 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:22.498 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:59:22.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:59:22.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:22.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:22.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:59:22.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:59:22.554 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:22.554 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:22.554 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:22.554 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:22.554 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:22.554 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:22.554 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:22.554 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:22.554 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:59:22.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: test_get_last_scrub_stamp: run_mgr td/ceph-helpers x 2026-02-18T08:59:22.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:59:22.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:59:22.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:59:22.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:59:22.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:59:22.608 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:59:22.723 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:59:22.723 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:22.723 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:22.723 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:22.723 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:22.723 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:22.723 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:22.724 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:22.725 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:22.744 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: test_get_last_scrub_stamp: run_osd td/ceph-helpers 0 2026-02-18T08:59:22.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T08:59:22.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:59:22.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:59:22.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:59:22.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:59:22.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:59:22.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:59:22.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:59:22.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:59:22.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:59:22.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:59:22.746 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:59:22.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:59:22.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:59:22.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:59:22.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:59:22.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:59:22.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:59:22.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:59:22.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:59:22.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:59:22.747 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:59:22.748 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:59:22.748 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=06d12f17-4791-4b4e-8f14-3b08a839e781 2026-02-18T08:59:22.748 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 06d12f17-4791-4b4e-8f14-3b08a839e781' 2026-02-18T08:59:22.748 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 06d12f17-4791-4b4e-8f14-3b08a839e781 2026-02-18T08:59:22.749 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:59:22.758 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBqf5Vpwy+PLBAAc+IoHW0garrgKORr87K1Kg== 2026-02-18T08:59:22.758 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBqf5Vpwy+PLBAAc+IoHW0garrgKORr87K1Kg=="}' 2026-02-18T08:59:22.759 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 06d12f17-4791-4b4e-8f14-3b08a839e781 -i td/ceph-helpers/0/new.json 2026-02-18T08:59:22.870 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:22.877 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T08:59:22.879 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBqf5Vpwy+PLBAAc+IoHW0garrgKORr87K1Kg== --osd-uuid 06d12f17-4791-4b4e-8f14-3b08a839e781 2026-02-18T08:59:22.895 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:22.882+0000 7f889c815a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:22.897 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:22.882+0000 7f889c815a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:22.897 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:22.886+0000 7f889c815a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:22.898 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:22.886+0000 7f889c815a40 -1 bdev(0x55766f0d2400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:59:22.898 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:22.886+0000 7f889c815a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T08:59:23.716 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T08:59:23.716 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:59:23.717 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:59:23.717 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:59:23.718 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:59:23.829 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:59:23.830 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:59:23.831 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:59:23.831 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:59:23.831 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:59:23.831 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:59:23.853 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:23.842+0000 7f52cad4fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:23.855 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:23.842+0000 7f52cad4fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:23.856 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:23.842+0000 7f52cad4fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:23.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:59:23.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:59:23.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:59:23.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:59:23.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:59:23.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:23.939 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:23.939 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:59:23.939 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:23.939 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:24.048 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:24.295 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:24.282+0000 7f52cad4fa40 -1 Falling back to public interface 2026-02-18T08:59:25.031 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:25.018+0000 7f52cad4fa40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:59:25.049 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:25.049 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:25.050 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:59:25.050 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:59:25.050 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:25.050 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:25.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:26.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:26.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:26.269 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:59:26.270 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:59:26.270 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:26.270 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:26.480 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2926074813,v1:127.0.0.1:6801/2926074813] [v2:127.0.0.1:6802/2926074813,v1:127.0.0.1:6803/2926074813] exists,up 06d12f17-4791-4b4e-8f14-3b08a839e781 2026-02-18T08:59:26.481 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:59:26.481 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:59:26.481 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:59:26.481 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1530: test_get_last_scrub_stamp: create_rbd_pool 2026-02-18T08:59:26.481 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T08:59:26.685 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T08:59:26.696 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T08:59:26.696 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T08:59:26.926 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T08:59:26.936 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T08:59:27.937 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T08:59:28.217 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1531: test_get_last_scrub_stamp: wait_for_clean 2026-02-18T08:59:28.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T08:59:28.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T08:59:28.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T08:59:28.218 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T08:59:28.219 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T08:59:28.219 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T08:59:28.219 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T08:59:28.219 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T08:59:28.219 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T08:59:28.306 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T08:59:28.306 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T08:59:28.306 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T08:59:28.307 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T08:59:28.307 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:59:28.307 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:59:28.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T08:59:28.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:59:28.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:59:28.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:59:28.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-18T08:59:28.610 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-18T08:59:28.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-18T08:59:28.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:59:28.611 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-18T08:59:28.611 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:59:28.612 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:59:28.612 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-18T08:59:28.612 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:59:28.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-18T08:59:28.614 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836483 2026-02-18T08:59:28.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-18T08:59:28.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:59:28.837 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-18T08:59:28.837 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:59:29.838 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:59:29.839 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:59:30.046 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-18T08:59:30.047 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T08:59:30.047 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:59:30.047 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:59:30.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T08:59:30.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T08:59:30.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T08:59:30.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T08:59:30.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T08:59:30.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T08:59:30.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T08:59:30.325 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T08:59:30.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T08:59:30.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T08:59:30.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:59:30.533 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:59:30.809 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T08:59:30.809 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T08:59:30.809 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T08:59:30.809 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: get_last_scrub_stamp 1.0 2026-02-18T08:59:30.810 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-18T08:59:30.810 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-18T08:59:30.810 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-18T08:59:30.810 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-18T08:59:31.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: stamp=2026-02-18T08:59:26.860970+0000 2026-02-18T08:59:31.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1533: test_get_last_scrub_stamp: test -n 2026-02-18T08:59:26.860970+0000 2026-02-18T08:59:31.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1534: test_get_last_scrub_stamp: teardown td/ceph-helpers 2026-02-18T08:59:31.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:31.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:31.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:31.020 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:31.020 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:31.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:31.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:31.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:31.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:31.128 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:31.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:31.129 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:31.130 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:31.130 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:31.130 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:31.131 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:31.131 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:31.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:31.132 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:31.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:31.134 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:31.134 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:31.138 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:31.138 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:31.138 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:31.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:31.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:31.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:31.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:31.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_not_primary td/ceph-helpers 2026-02-18T08:59:31.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: test_get_not_primary: local dir=td/ceph-helpers 2026-02-18T08:59:31.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1231: test_get_not_primary: setup td/ceph-helpers 2026-02-18T08:59:31.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:31.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:31.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:31.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:31.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:31.142 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:31.142 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:31.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:31.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:31.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:31.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:31.143 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:31.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:31.144 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:31.145 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:31.145 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:31.145 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:31.146 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:31.146 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:31.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:31.147 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:31.148 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:31.149 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:31.149 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:31.151 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:31.151 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:31.151 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:31.151 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:31.152 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:31.152 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:31.152 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:31.153 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:31.154 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:31.154 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:31.154 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:31.155 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:31.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:31.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:31.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:31.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: test_get_not_primary: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-18T08:59:31.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:59:31.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:59:31.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:59:31.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:59:31.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:59:31.157 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-18T08:59:31.177 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:59:31.177 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:31.177 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:31.178 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:31.178 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:31.178 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:31.178 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:31.178 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-18T08:59:31.204 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:59:31.205 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:59:31.205 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:31.205 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:31.205 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:59:31.207 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:59:31.207 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:31.207 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:31.207 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:31.207 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:31.207 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:31.207 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:31.207 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:31.207 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:31.207 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:59:31.261 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:59:31.261 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:31.261 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:31.261 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:59:31.261 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:59:31.262 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:31.263 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:31.263 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:31.263 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:31.263 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:31.263 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:31.263 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:31.263 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:31.263 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:59:31.316 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: test_get_not_primary: run_mgr td/ceph-helpers x 2026-02-18T08:59:31.316 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:59:31.316 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:59:31.316 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:59:31.316 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:59:31.317 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:59:31.317 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:59:31.425 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:59:31.426 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:31.426 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:31.426 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:31.426 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:31.427 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:31.427 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:31.427 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:31.428 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:31.448 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: test_get_not_primary: run_osd td/ceph-helpers 0 2026-02-18T08:59:31.448 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T08:59:31.448 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:59:31.448 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:59:31.449 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:59:31.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:59:31.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:59:31.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:59:31.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:59:31.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:59:31.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:59:31.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:59:31.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:59:31.452 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:59:31.452 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a70e3f58-71bf-4fce-85f7-965037f69453 2026-02-18T08:59:31.452 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a70e3f58-71bf-4fce-85f7-965037f69453' 2026-02-18T08:59:31.452 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 a70e3f58-71bf-4fce-85f7-965037f69453 2026-02-18T08:59:31.453 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:59:31.461 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBzf5Vpo6/UGhAA3C7YDaBpW3wl8leqM6w/TQ== 2026-02-18T08:59:31.461 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBzf5Vpo6/UGhAA3C7YDaBpW3wl8leqM6w/TQ=="}' 2026-02-18T08:59:31.461 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a70e3f58-71bf-4fce-85f7-965037f69453 -i td/ceph-helpers/0/new.json 2026-02-18T08:59:31.574 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:31.582 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T08:59:31.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBzf5Vpo6/UGhAA3C7YDaBpW3wl8leqM6w/TQ== --osd-uuid a70e3f58-71bf-4fce-85f7-965037f69453 2026-02-18T08:59:31.600 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:31.586+0000 7fed4cc98a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:31.601 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:31.590+0000 7fed4cc98a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:31.602 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:31.590+0000 7fed4cc98a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:31.603 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:31.590+0000 7fed4cc98a40 -1 bdev(0x564845a82400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:59:31.603 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:31.590+0000 7fed4cc98a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T08:59:32.374 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T08:59:32.374 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:59:32.375 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:59:32.375 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:59:32.375 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:59:32.493 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:59:32.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:59:32.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:59:32.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:59:32.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:59:32.495 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:59:32.509 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:32.498+0000 7f656645ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:32.511 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:32.498+0000 7f656645ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:32.512 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:32.498+0000 7f656645ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:32.601 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:59:32.602 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:59:32.602 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:59:32.602 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:59:32.602 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:59:32.602 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:32.602 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:32.603 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:59:32.603 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:32.603 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:32.706 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:33.031 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:33.018+0000 7f656645ca40 -1 Falling back to public interface 2026-02-18T08:59:33.708 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:33.708 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:33.708 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:59:33.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:59:33.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:33.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:33.820 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:33.994 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:33.982+0000 7f656645ca40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:59:34.822 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:34.822 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:34.823 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:59:34.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:59:34.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:34.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:35.032 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/947410813,v1:127.0.0.1:6801/947410813] [v2:127.0.0.1:6802/947410813,v1:127.0.0.1:6803/947410813] exists,up a70e3f58-71bf-4fce-85f7-965037f69453 2026-02-18T08:59:35.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:59:35.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:59:35.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:59:35.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1235: test_get_not_primary: run_osd td/ceph-helpers 1 2026-02-18T08:59:35.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T08:59:35.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:59:35.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-18T08:59:35.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:59:35.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-18T08:59:35.035 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:35.036 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:59:35.037 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:59:35.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:59:35.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:59:35.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:59:35.038 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-18T08:59:35.038 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:59:35.039 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=494c8193-a49c-4cd2-b634-cc483d8d372b 2026-02-18T08:59:35.039 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 494c8193-a49c-4cd2-b634-cc483d8d372b' 2026-02-18T08:59:35.039 INFO:tasks.workunit.client.0.trial118.stdout:add osd1 494c8193-a49c-4cd2-b634-cc483d8d372b 2026-02-18T08:59:35.039 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:59:35.048 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB3f5VpfYU7AhAAmYJh+aB7P/cuYjv/fDx+Dw== 2026-02-18T08:59:35.049 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB3f5VpfYU7AhAAmYJh+aB7P/cuYjv/fDx+Dw=="}' 2026-02-18T08:59:35.049 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 494c8193-a49c-4cd2-b634-cc483d8d372b -i td/ceph-helpers/1/new.json 2026-02-18T08:59:35.267 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:59:35.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-18T08:59:35.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB3f5VpfYU7AhAAmYJh+aB7P/cuYjv/fDx+Dw== --osd-uuid 494c8193-a49c-4cd2-b634-cc483d8d372b 2026-02-18T08:59:35.294 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:35.282+0000 7f8f72ea1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:35.296 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:35.282+0000 7f8f72ea1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:35.297 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:35.282+0000 7f8f72ea1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:35.298 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:35.286+0000 7f8f72ea1a40 -1 bdev(0x555c3dae4400 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-18T08:59:35.298 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:35.286+0000 7f8f72ea1a40 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-18T08:59:35.936 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-18T08:59:35.936 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:59:35.937 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-18T08:59:35.937 INFO:tasks.workunit.client.0.trial118.stdout:adding osd1 key to auth repository 2026-02-18T08:59:35.937 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:59:36.232 INFO:tasks.workunit.client.0.trial118.stdout:start osd.1 2026-02-18T08:59:36.232 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-18T08:59:36.233 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:59:36.233 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:59:36.233 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:59:36.233 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:59:36.247 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:36.234+0000 7f8838858a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:36.248 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:36.234+0000 7f8838858a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:36.249 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:36.238+0000 7f8838858a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:36.452 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-18T08:59:36.452 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:59:36.452 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-18T08:59:36.452 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:59:36.452 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:59:36.453 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:36.453 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:36.453 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:59:36.454 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:36.454 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:59:36.668 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:36.678 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:36.666+0000 7f8838858a40 -1 Falling back to public interface 2026-02-18T08:59:37.261 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:37.250+0000 7f8838858a40 -1 osd.1 0 log_to_monitors true 2026-02-18T08:59:37.669 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:37.669 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:37.669 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:59:37.670 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:59:37.670 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:37.670 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:59:37.888 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:38.890 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:38.890 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:38.890 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:59:38.891 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:59:38.891 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:38.891 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T08:59:39.103 INFO:tasks.workunit.client.0.trial118.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1429272494,v1:127.0.0.1:6811/1429272494] [v2:127.0.0.1:6812/1429272494,v1:127.0.0.1:6813/1429272494] exists,up 494c8193-a49c-4cd2-b634-cc483d8d372b 2026-02-18T08:59:39.103 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:59:39.103 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:59:39.103 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:59:39.103 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1236: test_get_not_primary: create_rbd_pool 2026-02-18T08:59:39.103 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T08:59:39.304 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T08:59:39.313 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T08:59:39.313 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T08:59:39.547 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T08:59:39.557 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T08:59:40.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T08:59:40.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1237: test_get_not_primary: wait_for_clean 2026-02-18T08:59:40.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T08:59:40.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T08:59:40.810 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T08:59:40.811 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T08:59:40.811 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T08:59:40.812 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T08:59:40.812 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T08:59:40.812 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T08:59:40.812 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T08:59:40.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T08:59:40.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T08:59:40.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T08:59:40.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T08:59:40.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:59:40.916 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:59:41.140 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-18T08:59:41.140 INFO:tasks.workunit.client.0.trial118.stderr:1' 2026-02-18T08:59:41.140 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:59:41.140 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:59:41.140 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:59:41.220 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-18T08:59:41.220 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-18T08:59:41.220 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-18T08:59:41.220 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:59:41.220 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-18T08:59:41.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-18T08:59:41.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-18T08:59:41.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-18T08:59:41.298 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:59:41.298 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-18T08:59:41.299 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:59:41.299 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:59:41.300 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-18T08:59:41.300 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:59:41.301 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-18T08:59:41.301 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836484 2026-02-18T08:59:41.301 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-18T08:59:41.302 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:59:41.521 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836484 2026-02-18T08:59:41.521 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:59:42.522 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:59:42.522 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:59:42.733 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-18T08:59:42.733 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:59:42.733 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705667 2026-02-18T08:59:42.734 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:59:42.735 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-18T08:59:42.736 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-18T08:59:42.736 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:59:42.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705667 2026-02-18T08:59:42.737 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-18T08:59:42.737 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.1 seq 38654705667 2026-02-18T08:59:42.738 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T08:59:42.951 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705667 -lt 38654705667 2026-02-18T08:59:42.952 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T08:59:42.952 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:59:42.952 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:59:43.233 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T08:59:43.234 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T08:59:43.234 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T08:59:43.234 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T08:59:43.234 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T08:59:43.234 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T08:59:43.234 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T08:59:43.234 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T08:59:43.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T08:59:43.443 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T08:59:43.443 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:59:43.444 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:59:43.726 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T08:59:43.726 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T08:59:43.726 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T08:59:43.726 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: get_primary rbd GROUP 2026-02-18T08:59:43.726 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-18T08:59:43.726 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-18T08:59:43.726 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-18T08:59:43.726 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-18T08:59:43.937 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: local primary=1 2026-02-18T08:59:43.937 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: get_not_primary rbd GROUP 2026-02-18T08:59:43.937 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=rbd 2026-02-18T08:59:43.937 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=GROUP 2026-02-18T08:59:43.938 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary rbd GROUP 2026-02-18T08:59:43.938 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-18T08:59:43.938 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-18T08:59:43.938 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-18T08:59:43.938 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-18T08:59:44.160 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-18T08:59:44.160 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map rbd GROUP 2026-02-18T08:59:44.161 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-18T08:59:44.379 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: local not_primary=0 2026-02-18T08:59:44.380 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1240: test_get_not_primary: test 0 '!=' 1 2026-02-18T08:59:44.380 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1241: test_get_not_primary: test 0 = 0 -o 0 = 1 2026-02-18T08:59:44.380 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1242: test_get_not_primary: teardown td/ceph-helpers 2026-02-18T08:59:44.380 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:44.380 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:44.380 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:44.380 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:44.380 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:44.380 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:44.381 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:44.381 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:44.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:44.489 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:44.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:44.490 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:44.491 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:44.491 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:44.491 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:44.492 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:44.493 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:44.493 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:44.493 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:44.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:44.495 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:44.495 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:44.501 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:44.501 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:44.502 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:44.502 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:44.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:44.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:44.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:44.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_active_clean td/ceph-helpers 2026-02-18T08:59:44.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: test_get_num_active_clean: local dir=td/ceph-helpers 2026-02-18T08:59:44.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: test_get_num_active_clean: setup td/ceph-helpers 2026-02-18T08:59:44.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:44.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:44.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:44.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:44.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:44.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:44.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:44.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:44.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:44.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:44.506 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:44.506 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:44.507 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:44.508 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:44.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:44.509 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:44.509 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:44.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:44.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:44.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:44.510 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:44.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:44.512 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:44.512 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:44.514 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:44.514 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:44.514 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:44.514 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:44.515 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:44.515 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:44.515 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:44.517 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:44.517 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:44.517 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:44.517 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:44.518 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:44.519 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:44.519 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:44.519 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:44.519 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: test_get_num_active_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:44.519 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:59:44.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:59:44.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:59:44.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:59:44.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:59:44.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:44.541 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:59:44.541 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:44.541 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:44.541 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:44.541 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:44.542 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:44.542 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:44.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:44.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:59:44.571 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:59:44.571 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:44.571 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:44.571 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:59:44.571 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:59:44.572 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:44.572 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:44.572 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:44.572 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:44.573 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:44.573 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:44.573 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:44.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:44.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:59:44.628 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:59:44.628 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:44.628 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:44.628 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:59:44.628 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:59:44.630 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:44.630 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:44.630 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:44.630 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:44.630 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:44.630 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:44.630 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:44.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:44.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:59:44.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: test_get_num_active_clean: run_mgr td/ceph-helpers x 2026-02-18T08:59:44.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:59:44.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:59:44.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:59:44.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:59:44.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:59:44.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:59:44.801 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:59:44.801 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:44.801 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:44.801 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:44.801 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:44.801 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:44.801 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:44.802 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:44.802 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:44.822 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: test_get_num_active_clean: run_osd td/ceph-helpers 0 2026-02-18T08:59:44.822 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T08:59:44.822 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:59:44.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:59:44.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:59:44.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:59:44.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:59:44.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:59:44.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:59:44.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:59:44.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:59:44.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:59:44.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:59:44.823 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:59:44.824 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:59:44.824 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:59:44.824 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:44.824 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:44.824 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:44.824 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:44.824 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:44.824 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:44.824 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:44.824 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:59:44.825 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:59:44.826 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9219a1a0-9b31-43fa-9403-c79f328b1605 2026-02-18T08:59:44.826 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9219a1a0-9b31-43fa-9403-c79f328b1605' 2026-02-18T08:59:44.827 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 9219a1a0-9b31-43fa-9403-c79f328b1605 2026-02-18T08:59:44.827 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:59:44.834 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCAf5VpJm4NMRAAYmsUDSLcEA6XhaTCBBkl2g== 2026-02-18T08:59:44.834 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCAf5VpJm4NMRAAYmsUDSLcEA6XhaTCBBkl2g=="}' 2026-02-18T08:59:44.834 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9219a1a0-9b31-43fa-9403-c79f328b1605 -i td/ceph-helpers/0/new.json 2026-02-18T08:59:44.943 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:44.950 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T08:59:44.952 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCAf5VpJm4NMRAAYmsUDSLcEA6XhaTCBBkl2g== --osd-uuid 9219a1a0-9b31-43fa-9403-c79f328b1605 2026-02-18T08:59:44.967 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:44.954+0000 7f67efa90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:44.968 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:44.954+0000 7f67efa90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:44.969 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:44.958+0000 7f67efa90a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:44.970 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:44.958+0000 7f67efa90a40 -1 bdev(0x560e45278400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:59:44.970 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:44.958+0000 7f67efa90a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T08:59:45.701 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T08:59:45.701 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:59:45.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:59:45.702 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:59:45.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:59:45.820 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:59:45.821 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:59:45.821 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:59:45.821 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:59:45.821 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:59:45.821 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:59:45.846 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:45.834+0000 7f3faaab9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:45.848 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:45.834+0000 7f3faaab9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:45.849 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:45.834+0000 7f3faaab9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:45.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:59:45.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:59:45.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:59:45.932 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:59:45.933 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:59:45.933 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:45.933 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:59:45.933 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:45.934 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:45.934 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:46.040 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:46.355 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:46.342+0000 7f3faaab9a40 -1 Falling back to public interface 2026-02-18T08:59:47.041 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:47.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:47.042 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:59:47.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:59:47.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:47.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:47.152 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:47.267 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:47.254+0000 7f3faaab9a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:59:48.154 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:48.154 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:48.155 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:59:48.155 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:59:48.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:48.156 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:48.372 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2516760970,v1:127.0.0.1:6801/2516760970] [v2:127.0.0.1:6802/2516760970,v1:127.0.0.1:6803/2516760970] exists,up 9219a1a0-9b31-43fa-9403-c79f328b1605 2026-02-18T08:59:48.373 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:59:48.374 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:59:48.374 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:59:48.374 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1369: test_get_num_active_clean: create_rbd_pool 2026-02-18T08:59:48.374 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T08:59:48.583 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T08:59:48.592 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T08:59:48.592 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T08:59:48.876 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T08:59:48.885 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T08:59:49.886 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T08:59:50.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1370: test_get_num_active_clean: wait_for_clean 2026-02-18T08:59:50.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T08:59:50.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T08:59:50.183 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T08:59:50.183 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T08:59:50.183 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T08:59:50.183 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T08:59:50.183 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T08:59:50.184 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T08:59:50.184 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T08:59:50.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T08:59:50.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T08:59:50.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T08:59:50.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T08:59:50.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:59:50.278 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:59:50.499 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T08:59:50.499 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:59:50.499 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:59:50.500 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:59:50.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-18T08:59:50.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-18T08:59:50.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-18T08:59:50.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:59:50.581 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-18T08:59:50.581 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:59:50.581 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:59:50.582 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-18T08:59:50.582 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:59:50.583 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-18T08:59:50.584 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836483 2026-02-18T08:59:50.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-18T08:59:50.584 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:59:50.800 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-18T08:59:50.800 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T08:59:51.801 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T08:59:51.802 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:59:52.021 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-18T08:59:52.022 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T08:59:52.023 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:59:52.023 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:59:52.308 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T08:59:52.308 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T08:59:52.308 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T08:59:52.308 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T08:59:52.308 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T08:59:52.308 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T08:59:52.308 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T08:59:52.308 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T08:59:52.517 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T08:59:52.517 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T08:59:52.518 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T08:59:52.518 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T08:59:52.799 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T08:59:52.799 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T08:59:52.799 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T08:59:52.799 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: get_num_active_clean 2026-02-18T08:59:52.800 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T08:59:52.800 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T08:59:52.800 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T08:59:52.800 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T08:59:52.800 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T08:59:53.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: local num_active_clean=4 2026-02-18T08:59:53.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1372: test_get_num_active_clean: test 4 = 4 2026-02-18T08:59:53.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1373: test_get_num_active_clean: teardown td/ceph-helpers 2026-02-18T08:59:53.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:53.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:53.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:53.011 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:53.011 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:53.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:53.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:53.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:53.118 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:53.118 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:53.119 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:53.119 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:53.120 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:53.120 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:53.121 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:53.122 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:53.122 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:53.122 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:53.122 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:53.123 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:53.124 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:53.124 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:53.129 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:53.129 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:53.129 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:53.130 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:53.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:53.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:53.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T08:59:53.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_active_or_peered td/ceph-helpers 2026-02-18T08:59:53.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1393: test_get_num_active_or_peered: local dir=td/ceph-helpers 2026-02-18T08:59:53.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1395: test_get_num_active_or_peered: setup td/ceph-helpers 2026-02-18T08:59:53.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T08:59:53.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T08:59:53.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T08:59:53.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T08:59:53.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T08:59:53.133 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T08:59:53.133 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T08:59:53.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T08:59:53.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T08:59:53.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T08:59:53.134 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T08:59:53.134 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T08:59:53.135 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T08:59:53.135 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T08:59:53.136 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T08:59:53.136 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T08:59:53.136 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T08:59:53.137 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:53.137 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T08:59:53.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T08:59:53.138 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T08:59:53.139 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T08:59:53.140 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T08:59:53.140 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T08:59:53.141 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T08:59:53.141 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:53.142 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:53.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T08:59:53.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T08:59:53.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T08:59:53.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T08:59:53.144 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T08:59:53.144 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:53.144 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:53.144 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T08:59:53.146 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T08:59:53.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T08:59:53.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T08:59:53.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T08:59:53.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1396: test_get_num_active_or_peered: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:53.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T08:59:53.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T08:59:53.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T08:59:53.147 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T08:59:53.148 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T08:59:53.148 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:53.169 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T08:59:53.169 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:53.169 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:53.169 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:53.169 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:53.169 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:53.169 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:53.170 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T08:59:53.197 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T08:59:53.198 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T08:59:53.198 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:53.198 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:53.198 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T08:59:53.198 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T08:59:53.199 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:53.199 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:53.199 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:53.199 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:53.199 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:53.199 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:53.199 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:53.200 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:53.200 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T08:59:53.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T08:59:53.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T08:59:53.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T08:59:53.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T08:59:53.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T08:59:53.255 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T08:59:53.255 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T08:59:53.256 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T08:59:53.257 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T08:59:53.257 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:53.257 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:53.257 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T08:59:53.257 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T08:59:53.257 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T08:59:53.309 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1397: test_get_num_active_or_peered: run_mgr td/ceph-helpers x 2026-02-18T08:59:53.309 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T08:59:53.310 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T08:59:53.310 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T08:59:53.310 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T08:59:53.310 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T08:59:53.310 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T08:59:53.416 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T08:59:53.416 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:53.416 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:53.416 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:53.416 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:53.416 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:53.416 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:53.417 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:53.417 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T08:59:53.440 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1398: test_get_num_active_or_peered: run_osd td/ceph-helpers 0 2026-02-18T08:59:53.440 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T08:59:53.440 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T08:59:53.440 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T08:59:53.440 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T08:59:53.440 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T08:59:53.440 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T08:59:53.441 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T08:59:53.441 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T08:59:53.441 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T08:59:53.441 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T08:59:53.441 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T08:59:53.441 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T08:59:53.441 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T08:59:53.441 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T08:59:53.441 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T08:59:53.441 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T08:59:53.441 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T08:59:53.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T08:59:53.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T08:59:53.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T08:59:53.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T08:59:53.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T08:59:53.443 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T08:59:53.444 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=88e85a54-bfc0-40d5-9ec7-85ecfd67e053 2026-02-18T08:59:53.444 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 88e85a54-bfc0-40d5-9ec7-85ecfd67e053 2026-02-18T08:59:53.444 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 88e85a54-bfc0-40d5-9ec7-85ecfd67e053' 2026-02-18T08:59:53.444 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T08:59:53.454 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCJf5Vp/VdrGhAAoWAAxmXDdik4Urh4ahLJew== 2026-02-18T08:59:53.454 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCJf5Vp/VdrGhAAoWAAxmXDdik4Urh4ahLJew=="}' 2026-02-18T08:59:53.454 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 88e85a54-bfc0-40d5-9ec7-85ecfd67e053 -i td/ceph-helpers/0/new.json 2026-02-18T08:59:53.569 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:53.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T08:59:53.577 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCJf5Vp/VdrGhAAoWAAxmXDdik4Urh4ahLJew== --osd-uuid 88e85a54-bfc0-40d5-9ec7-85ecfd67e053 2026-02-18T08:59:53.594 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:53.582+0000 7fe0da229a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:53.596 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:53.582+0000 7fe0da229a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:53.596 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:53.582+0000 7fe0da229a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:53.597 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:53.582+0000 7fe0da229a40 -1 bdev(0x563ea3606400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T08:59:53.597 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:53.582+0000 7fe0da229a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T08:59:54.341 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T08:59:54.341 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T08:59:54.342 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T08:59:54.343 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T08:59:54.343 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T08:59:54.462 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T08:59:54.463 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T08:59:54.463 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T08:59:54.463 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T08:59:54.464 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T08:59:54.464 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T08:59:54.486 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:54.474+0000 7f72d7e31a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:54.488 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:54.474+0000 7f72d7e31a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:54.490 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:54.478+0000 7f72d7e31a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T08:59:54.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T08:59:54.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T08:59:54.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T08:59:54.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T08:59:54.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T08:59:54.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:54.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T08:59:54.572 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T08:59:54.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:54.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:54.677 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:54.947 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:54.934+0000 7f72d7e31a40 -1 Falling back to public interface 2026-02-18T08:59:55.659 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T08:59:55.646+0000 7f72d7e31a40 -1 osd.0 0 log_to_monitors true 2026-02-18T08:59:55.678 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:55.678 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:55.678 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T08:59:55.679 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T08:59:55.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:55.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:55.785 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T08:59:56.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T08:59:56.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T08:59:56.787 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T08:59:56.788 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T08:59:56.788 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T08:59:56.788 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T08:59:57.005 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2255895719,v1:127.0.0.1:6801/2255895719] [v2:127.0.0.1:6802/2255895719,v1:127.0.0.1:6803/2255895719] exists,up 88e85a54-bfc0-40d5-9ec7-85ecfd67e053 2026-02-18T08:59:57.006 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T08:59:57.007 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T08:59:57.007 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T08:59:57.007 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1399: test_get_num_active_or_peered: create_rbd_pool 2026-02-18T08:59:57.007 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T08:59:57.217 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T08:59:57.225 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T08:59:57.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T08:59:57.503 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T08:59:57.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T08:59:58.515 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T08:59:58.802 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1400: test_get_num_active_or_peered: wait_for_clean 2026-02-18T08:59:58.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T08:59:58.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T08:59:58.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T08:59:58.803 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T08:59:58.803 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T08:59:58.803 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T08:59:58.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T08:59:58.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T08:59:58.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T08:59:58.903 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T08:59:58.903 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T08:59:58.903 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T08:59:58.904 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T08:59:58.904 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T08:59:58.904 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T08:59:59.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T08:59:59.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T08:59:59.127 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T08:59:59.128 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T08:59:59.207 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-18T08:59:59.207 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-18T08:59:59.207 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-18T08:59:59.207 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T08:59:59.208 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-18T08:59:59.208 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T08:59:59.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T08:59:59.209 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-18T08:59:59.209 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T08:59:59.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-18T08:59:59.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-18T08:59:59.211 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836483 2026-02-18T08:59:59.211 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T08:59:59.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-18T08:59:59.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:00:00.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:00:00.438 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:00:00.648 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-18T09:00:00.649 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T09:00:00.649 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:00.650 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:00.936 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T09:00:00.936 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T09:00:00.936 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T09:00:00.936 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:00:00.937 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:00:00.937 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:00:00.937 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:00:00.937 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:00:01.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T09:00:01.141 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T09:00:01.142 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:01.142 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:01.433 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T09:00:01.433 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T09:00:01.433 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T09:00:01.433 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: get_num_active_or_peered 2026-02-18T09:00:01.433 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-18T09:00:01.433 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-18T09:00:01.433 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-18T09:00:01.433 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-18T09:00:01.433 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-18T09:00:01.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: local num_peered=4 2026-02-18T09:00:01.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1402: test_get_num_active_or_peered: test 4 = 4 2026-02-18T09:00:01.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1403: test_get_num_active_or_peered: teardown td/ceph-helpers 2026-02-18T09:00:01.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:01.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:01.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:01.646 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:01.646 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:01.647 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:01.647 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:01.647 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:01.754 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:01.755 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:01.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:01.756 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:01.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:01.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:01.758 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:01.759 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:01.759 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:01.759 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:01.759 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:01.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:01.761 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:01.761 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:01.765 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:01.765 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:01.765 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:01.765 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_pgs td/ceph-helpers 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1420: test_get_num_pgs: local dir=td/ceph-helpers 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1422: test_get_num_pgs: setup td/ceph-helpers 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:01.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:01.770 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:01.770 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:01.771 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:01.771 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:01.772 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:01.772 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:01.772 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:01.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:01.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:01.774 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:01.774 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:01.775 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:01.776 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:01.776 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:01.777 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:01.777 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:01.777 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:01.778 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:01.778 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:01.779 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:01.779 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:00:01.780 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:00:01.780 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:01.780 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:01.780 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:00:01.781 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:00:01.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:00:01.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:00:01.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:00:01.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1423: test_get_num_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:01.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:00:01.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:00:01.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:00:01.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:00:01.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:00:01.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:01.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:00:01.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:01.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:01.805 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:01.805 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:01.805 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:01.805 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:01.806 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:01.832 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:00:01.833 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:00:01.833 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:01.833 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:01.833 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:00:01.833 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:00:01.835 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:01.835 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:01.835 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:01.835 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:01.835 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:01.835 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:01.835 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:01.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:01.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:00:01.890 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:00:01.891 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:01.891 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:01.891 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:00:01.891 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:00:01.891 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:01.891 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:01.891 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:01.892 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:01.892 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:01.892 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:01.893 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:01.893 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:01.893 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:00:01.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1424: test_get_num_pgs: run_mgr td/ceph-helpers x 2026-02-18T09:00:01.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:00:01.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:00:01.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:00:01.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:00:01.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:00:01.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:00:02.060 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:00:02.060 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:02.060 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:02.060 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:02.060 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:02.061 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:02.061 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:02.061 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:02.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:02.084 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: test_get_num_pgs: run_osd td/ceph-helpers 0 2026-02-18T09:00:02.084 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:00:02.084 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:00:02.084 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:00:02.085 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:00:02.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:00:02.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:00:02.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:00:02.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:00:02.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:00:02.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:00:02.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:00:02.087 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:00:02.088 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2f96c974-5cc7-4686-8970-a92b804b345c 2026-02-18T09:00:02.088 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2f96c974-5cc7-4686-8970-a92b804b345c' 2026-02-18T09:00:02.088 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 2f96c974-5cc7-4686-8970-a92b804b345c 2026-02-18T09:00:02.088 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:00:02.096 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCSf5VpRWURBRAAQZ02woMNZo9h9tK01O6+1w== 2026-02-18T09:00:02.097 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCSf5VpRWURBRAAQZ02woMNZo9h9tK01O6+1w=="}' 2026-02-18T09:00:02.097 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2f96c974-5cc7-4686-8970-a92b804b345c -i td/ceph-helpers/0/new.json 2026-02-18T09:00:02.205 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:02.213 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:00:02.214 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCSf5VpRWURBRAAQZ02woMNZo9h9tK01O6+1w== --osd-uuid 2f96c974-5cc7-4686-8970-a92b804b345c 2026-02-18T09:00:02.229 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:02.218+0000 7f2944849a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:02.231 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:02.218+0000 7f2944849a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:02.232 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:02.218+0000 7f2944849a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:02.233 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:02.218+0000 7f2944849a40 -1 bdev(0x562d3cfe4400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:00:02.233 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:02.218+0000 7f2944849a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:00:02.993 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:00:02.993 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:00:02.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:00:02.994 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:00:02.995 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:00:03.116 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:00:03.117 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:00:03.118 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:00:03.118 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:00:03.118 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:00:03.118 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:00:03.142 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:03.129+0000 7f99d1b9ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:03.145 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:03.129+0000 7f99d1b9ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:03.146 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:03.133+0000 7f99d1b9ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:03.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:00:03.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:00:03.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:00:03.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:00:03.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:00:03.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:03.228 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:03.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:00:03.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:03.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:03.338 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:03.671 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:03.657+0000 7f99d1b9ba40 -1 Falling back to public interface 2026-02-18T09:00:04.339 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:04.339 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:04.340 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:00:04.340 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:00:04.340 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:04.341 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:04.448 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:04.607 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:04.593+0000 7f99d1b9ba40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:00:05.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:05.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:05.450 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:00:05.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:00:05.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:05.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:05.662 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2925047933,v1:127.0.0.1:6801/2925047933] [v2:127.0.0.1:6802/2925047933,v1:127.0.0.1:6803/2925047933] exists,up 2f96c974-5cc7-4686-8970-a92b804b345c 2026-02-18T09:00:05.663 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:00:05.663 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:00:05.663 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:00:05.663 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1426: test_get_num_pgs: create_rbd_pool 2026-02-18T09:00:05.663 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T09:00:05.867 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T09:00:05.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T09:00:05.876 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T09:00:06.109 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T09:00:06.118 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T09:00:07.119 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T09:00:07.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1427: test_get_num_pgs: wait_for_clean 2026-02-18T09:00:07.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T09:00:07.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T09:00:07.445 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T09:00:07.446 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T09:00:07.446 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:07.447 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:07.447 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:07.447 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:07.447 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:07.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T09:00:07.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T09:00:07.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T09:00:07.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T09:00:07.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:00:07.544 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:00:07.755 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:00:07.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:00:07.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:00:07.756 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:00:07.836 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-18T09:00:07.837 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-18T09:00:07.837 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-18T09:00:07.837 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:00:07.837 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-18T09:00:07.837 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:00:07.837 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:00:07.838 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-18T09:00:07.838 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:00:07.839 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-18T09:00:07.839 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836483 2026-02-18T09:00:07.840 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-18T09:00:07.840 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:00:08.062 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-18T09:00:08.062 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:00:09.063 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:00:09.063 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:00:09.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-18T09:00:09.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T09:00:09.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:09.284 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:09.577 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T09:00:09.578 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T09:00:09.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T09:00:09.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:00:09.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:00:09.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:00:09.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:00:09.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:00:09.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T09:00:09.787 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T09:00:09.788 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:09.788 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:10.072 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T09:00:10.072 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T09:00:10.072 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T09:00:10.072 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: get_num_pgs 2026-02-18T09:00:10.073 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:10.073 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:10.367 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: local num_pgs=4 2026-02-18T09:00:10.367 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1429: test_get_num_pgs: test 4 -gt 0 2026-02-18T09:00:10.367 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1430: test_get_num_pgs: teardown td/ceph-helpers 2026-02-18T09:00:10.367 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:10.367 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:10.368 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:10.368 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:10.368 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:10.368 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:10.368 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:10.368 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:10.475 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:10.476 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:10.477 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:10.477 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:10.478 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:10.478 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:10.479 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:10.480 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:10.480 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:10.480 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:10.480 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:10.481 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:10.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:10.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:10.486 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:10.486 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:10.486 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:10.486 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:10.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:10.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:10.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:00:10.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_op_scheduler td/ceph-helpers 2026-02-18T09:00:10.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2315: test_get_op_scheduler: local dir=td/ceph-helpers 2026-02-18T09:00:10.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2317: test_get_op_scheduler: setup td/ceph-helpers 2026-02-18T09:00:10.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:00:10.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:00:10.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:10.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:10.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:10.490 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:10.491 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:10.491 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:10.491 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:10.491 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:10.492 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:10.492 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:10.492 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:10.492 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:10.493 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:10.493 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:10.493 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:10.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:10.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:10.495 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:10.495 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:10.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:10.496 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:10.497 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:10.498 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:10.498 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:10.498 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:10.498 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:10.499 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:10.499 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:10.500 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:00:10.501 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:00:10.501 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:10.501 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:10.501 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:00:10.502 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:00:10.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:00:10.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:00:10.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:00:10.503 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2319: test_get_op_scheduler: run_mon td/ceph-helpers a 2026-02-18T09:00:10.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:00:10.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:00:10.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:00:10.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:00:10.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:00:10.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-18T09:00:10.524 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:00:10.524 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:10.525 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:10.525 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:10.525 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:10.525 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:10.525 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:10.526 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-18T09:00:10.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:00:10.555 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:00:10.555 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:10.555 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:10.555 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:00:10.555 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:00:10.556 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:10.556 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:10.556 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:10.556 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:10.556 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:10.556 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:10.556 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:10.557 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:10.557 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:00:10.611 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:00:10.612 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:10.612 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:10.612 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:00:10.612 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:00:10.612 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:10.612 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:10.612 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:10.612 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:10.612 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:10.612 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:10.613 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:10.613 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:10.613 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:00:10.666 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2320: test_get_op_scheduler: run_mgr td/ceph-helpers x 2026-02-18T09:00:10.666 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:00:10.666 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:00:10.666 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:00:10.666 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:00:10.666 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:00:10.666 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:00:10.783 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:00:10.783 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:10.783 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:10.784 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:10.784 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:10.784 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:10.784 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:10.784 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:10.785 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2322: test_get_op_scheduler: run_osd td/ceph-helpers 0 --osd_op_queue=wpq 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:00:10.803 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=wpq 2026-02-18T09:00:10.804 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:00:10.805 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:00:10.806 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=62de1bd7-e3ba-421c-bb86-c72b2f60630e 2026-02-18T09:00:10.806 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 62de1bd7-e3ba-421c-bb86-c72b2f60630e 2026-02-18T09:00:10.806 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 62de1bd7-e3ba-421c-bb86-c72b2f60630e' 2026-02-18T09:00:10.806 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:00:10.818 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCaf5Vpxg4kMBAAAH6oQ6xnl6n5mEuhgFvewA== 2026-02-18T09:00:10.819 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCaf5Vpxg4kMBAAAH6oQ6xnl6n5mEuhgFvewA=="}' 2026-02-18T09:00:10.819 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 62de1bd7-e3ba-421c-bb86-c72b2f60630e -i td/ceph-helpers/0/new.json 2026-02-18T09:00:10.935 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:10.943 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:00:10.945 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq --mkfs --key AQCaf5Vpxg4kMBAAAH6oQ6xnl6n5mEuhgFvewA== --osd-uuid 62de1bd7-e3ba-421c-bb86-c72b2f60630e 2026-02-18T09:00:10.960 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:10.945+0000 7f6040cf9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:10.962 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:10.949+0000 7f6040cf9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:10.963 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:10.949+0000 7f6040cf9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:10.964 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:10.949+0000 7f6040cf9a40 -1 bdev(0x55d08478c400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:00:10.964 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:10.949+0000 7f6040cf9a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:00:11.783 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:00:11.783 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:00:11.784 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:00:11.785 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:00:11.785 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:00:11.905 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:00:11.906 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:00:11.906 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq 2026-02-18T09:00:11.906 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:00:11.906 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:00:11.906 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:00:11.931 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:11.917+0000 7ff436ae7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:11.933 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:11.917+0000 7ff436ae7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:11.934 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:11.921+0000 7ff436ae7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:12.014 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:00:12.015 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:00:12.015 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:00:12.015 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:00:12.015 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:00:12.015 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:12.015 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:00:12.016 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:12.016 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:12.016 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:12.120 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:12.463 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:12.449+0000 7ff436ae7a40 -1 Falling back to public interface 2026-02-18T09:00:13.122 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:13.122 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:13.123 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:00:13.123 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:00:13.124 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:13.124 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:13.336 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:13.372 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:13.357+0000 7ff436ae7a40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:00:14.337 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:14.337 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:14.337 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:00:14.338 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:00:14.338 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:14.338 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:14.546 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/149028662,v1:127.0.0.1:6801/149028662] [v2:127.0.0.1:6802/149028662,v1:127.0.0.1:6803/149028662] exists,up 62de1bd7-e3ba-421c-bb86-c72b2f60630e 2026-02-18T09:00:14.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:00:14.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:00:14.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:00:14.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2323: test_get_op_scheduler: get_op_scheduler 0 2026-02-18T09:00:14.550 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: get_op_scheduler: local id=0 2026-02-18T09:00:14.550 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2311: get_op_scheduler: get_config osd 0 osd_op_queue 2026-02-18T09:00:14.550 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-18T09:00:14.550 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-18T09:00:14.550 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-18T09:00:14.550 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-18T09:00:14.550 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-18T09:00:14.550 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-18T09:00:14.550 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-18T09:00:14.550 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:14.551 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:14.551 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:14.551 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-osd.0.asok 2026-02-18T09:00:14.551 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:14.551 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-osd.0.asok config get osd_op_queue 2026-02-18T09:00:14.616 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2323: test_get_op_scheduler: test wpq = wpq 2026-02-18T09:00:14.617 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2325: test_get_op_scheduler: run_osd td/ceph-helpers 1 --osd_op_queue=mclock_scheduler 2026-02-18T09:00:14.617 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:00:14.617 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:00:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-18T09:00:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:00:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-18T09:00:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:00:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:00:14.618 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:00:14.619 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:00:14.619 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-18T09:00:14.619 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-18T09:00:14.619 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:00:14.619 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:00:14.620 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:00:14.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:00:14.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:14.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:14.620 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:14.620 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:14.621 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:14.621 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:14.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:14.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:00:14.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:00:14.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:00:14.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:00:14.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:00:14.621 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:00:14.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:00:14.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:00:14.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:00:14.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:00:14.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-18T09:00:14.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-18T09:00:14.622 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:00:14.622 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=787f4de9-d04c-4e64-8ecd-5b2e3b6ba82d 2026-02-18T09:00:14.623 INFO:tasks.workunit.client.0.trial118.stdout:add osd1 787f4de9-d04c-4e64-8ecd-5b2e3b6ba82d 2026-02-18T09:00:14.623 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 787f4de9-d04c-4e64-8ecd-5b2e3b6ba82d' 2026-02-18T09:00:14.624 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:00:14.627 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCef5VpytqwJBAARLkMzJCrc262eW8zO00vAQ== 2026-02-18T09:00:14.627 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCef5VpytqwJBAARLkMzJCrc262eW8zO00vAQ=="}' 2026-02-18T09:00:14.628 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 787f4de9-d04c-4e64-8ecd-5b2e3b6ba82d -i td/ceph-helpers/1/new.json 2026-02-18T09:00:14.845 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:00:14.854 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-18T09:00:14.855 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQCef5VpytqwJBAARLkMzJCrc262eW8zO00vAQ== --osd-uuid 787f4de9-d04c-4e64-8ecd-5b2e3b6ba82d 2026-02-18T09:00:14.871 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:14.857+0000 7f98a78c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:14.872 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:14.857+0000 7f98a78c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:14.873 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:14.861+0000 7f98a78c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:14.874 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:14.861+0000 7f98a78c8a40 -1 bdev(0x55c285e38400 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-18T09:00:14.874 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:14.861+0000 7f98a78c8a40 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-18T09:00:15.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-18T09:00:15.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:00:15.608 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-18T09:00:15.608 INFO:tasks.workunit.client.0.trial118.stdout:adding osd1 key to auth repository 2026-02-18T09:00:15.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:00:15.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-18T09:00:15.909 INFO:tasks.workunit.client.0.trial118.stdout:start osd.1 2026-02-18T09:00:15.909 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-18T09:00:15.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:00:15.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:00:15.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:00:15.924 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:15.909+0000 7fbc5ea1da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:15.925 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:15.909+0000 7fbc5ea1da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:15.926 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:15.913+0000 7fbc5ea1da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:16.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-18T09:00:16.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:00:16.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-18T09:00:16.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:00:16.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:00:16.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:16.129 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:16.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:00:16.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:16.129 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T09:00:16.299 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:16.285+0000 7fbc5ea1da40 -1 Falling back to public interface 2026-02-18T09:00:16.342 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:16.924 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:16.909+0000 7fbc5ea1da40 -1 osd.1 0 log_to_monitors true 2026-02-18T09:00:17.344 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:17.344 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:17.344 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:00:17.344 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:00:17.345 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:17.345 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T09:00:17.561 INFO:tasks.workunit.client.0.trial118.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1022767888,v1:127.0.0.1:6811/1022767888] [v2:127.0.0.1:6812/1022767888,v1:127.0.0.1:6813/1022767888] exists,up 787f4de9-d04c-4e64-8ecd-5b2e3b6ba82d 2026-02-18T09:00:17.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:00:17.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:00:17.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:00:17.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: test_get_op_scheduler: get_op_scheduler 1 2026-02-18T09:00:17.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: get_op_scheduler: local id=1 2026-02-18T09:00:17.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2311: get_op_scheduler: get_config osd 1 osd_op_queue 2026-02-18T09:00:17.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-18T09:00:17.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=1 2026-02-18T09:00:17.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-18T09:00:17.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-18T09:00:17.565 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.1 2026-02-18T09:00:17.565 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-18T09:00:17.565 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-18T09:00:17.565 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:17.565 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:17.565 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:17.565 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-osd.1.asok 2026-02-18T09:00:17.565 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:17.565 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-osd.1.asok config get osd_op_queue 2026-02-18T09:00:17.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: test_get_op_scheduler: test mclock_scheduler = mclock_scheduler 2026-02-18T09:00:17.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2327: test_get_op_scheduler: teardown td/ceph-helpers 2026-02-18T09:00:17.643 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:17.643 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:17.643 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:17.643 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:17.643 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:17.644 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:17.644 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:17.644 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:17.749 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:17.749 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:17.750 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:17.751 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:17.752 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:17.752 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:17.753 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:17.753 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:17.754 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:17.754 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:17.755 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:17.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:17.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:17.761 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:17.761 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:17.761 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:17.762 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:00:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_osd_id_used_by_pgs td/ceph-helpers 2026-02-18T09:00:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1450: test_get_osd_id_used_by_pgs: local dir=td/ceph-helpers 2026-02-18T09:00:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1452: test_get_osd_id_used_by_pgs: setup td/ceph-helpers 2026-02-18T09:00:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:00:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:00:17.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:17.764 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:17.764 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:17.764 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:17.764 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:17.764 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:17.764 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:17.764 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:17.766 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:17.766 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:17.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:17.768 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:17.769 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:17.769 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:17.769 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:17.770 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:17.770 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:17.770 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:17.771 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:17.772 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:17.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:17.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:17.774 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:17.774 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:17.774 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:17.774 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:17.776 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:17.776 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:17.776 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:00:17.777 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:00:17.777 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:17.778 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:17.778 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:00:17.779 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:00:17.780 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:00:17.780 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:00:17.780 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:00:17.781 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1453: test_get_osd_id_used_by_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:17.781 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:00:17.781 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:00:17.781 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:00:17.781 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:00:17.781 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:00:17.781 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:17.802 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:00:17.802 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:17.802 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:17.802 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:17.803 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:17.803 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:17.803 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:17.803 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:17.833 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:00:17.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:00:17.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:17.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:17.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:00:17.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:00:17.835 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:17.836 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:17.836 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:17.836 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:17.836 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:17.836 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:17.836 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:17.836 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:17.837 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:00:17.892 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:00:17.892 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:17.892 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:17.892 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:00:17.892 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:00:17.892 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:17.892 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:17.893 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:17.893 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:17.893 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:17.894 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:17.894 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:17.894 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:17.894 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:00:17.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1454: test_get_osd_id_used_by_pgs: run_mgr td/ceph-helpers x 2026-02-18T09:00:17.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:00:17.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:00:17.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:00:17.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:00:17.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:00:17.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:00:18.063 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:00:18.064 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:18.064 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:18.064 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:18.064 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:18.064 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:18.064 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:18.065 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:18.065 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:18.084 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1455: test_get_osd_id_used_by_pgs: run_osd td/ceph-helpers 0 2026-02-18T09:00:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:00:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:00:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:00:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:00:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:00:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:00:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:00:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:00:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:00:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:18.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:18.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:00:18.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:00:18.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:00:18.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:00:18.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:00:18.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:00:18.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:00:18.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:00:18.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:00:18.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:00:18.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:00:18.088 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:00:18.088 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:00:18.088 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9b5e7b23-8703-4d48-9652-d9a267ae1674 2026-02-18T09:00:18.089 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9b5e7b23-8703-4d48-9652-d9a267ae1674' 2026-02-18T09:00:18.089 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 9b5e7b23-8703-4d48-9652-d9a267ae1674 2026-02-18T09:00:18.089 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:00:18.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCif5Vpo1kvBRAASQRJ9hOqx6e5OkUDEK8d4w== 2026-02-18T09:00:18.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCif5Vpo1kvBRAASQRJ9hOqx6e5OkUDEK8d4w=="}' 2026-02-18T09:00:18.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9b5e7b23-8703-4d48-9652-d9a267ae1674 -i td/ceph-helpers/0/new.json 2026-02-18T09:00:18.208 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:18.216 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:00:18.217 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCif5Vpo1kvBRAASQRJ9hOqx6e5OkUDEK8d4w== --osd-uuid 9b5e7b23-8703-4d48-9652-d9a267ae1674 2026-02-18T09:00:18.232 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:18.217+0000 7f103c9aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:18.233 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:18.221+0000 7f103c9aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:18.234 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:18.221+0000 7f103c9aba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:18.235 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:18.221+0000 7f103c9aba40 -1 bdev(0x55e255b08400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:00:18.235 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:18.221+0000 7f103c9aba40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:00:19.041 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:00:19.041 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:00:19.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:00:19.043 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:00:19.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:00:19.167 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:00:19.168 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:00:19.168 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:00:19.169 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:00:19.169 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:00:19.169 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:00:19.193 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:19.177+0000 7f7ce9a2ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:19.195 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:19.181+0000 7f7ce9a2ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:19.196 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:19.181+0000 7f7ce9a2ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:19.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:00:19.280 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:00:19.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:00:19.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:00:19.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:00:19.281 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:19.281 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:19.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:00:19.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:19.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:19.391 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:19.688 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:19.673+0000 7f7ce9a2ba40 -1 Falling back to public interface 2026-02-18T09:00:20.392 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:20.392 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:20.393 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:00:20.393 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:00:20.394 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:20.394 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:20.587 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:20.573+0000 7f7ce9a2ba40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:00:20.607 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:21.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:21.609 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:21.610 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:00:21.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:00:21.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:21.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:21.826 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1973962418,v1:127.0.0.1:6801/1973962418] [v2:127.0.0.1:6802/1973962418,v1:127.0.0.1:6803/1973962418] exists,up 9b5e7b23-8703-4d48-9652-d9a267ae1674 2026-02-18T09:00:21.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:00:21.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:00:21.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:00:21.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1456: test_get_osd_id_used_by_pgs: create_rbd_pool 2026-02-18T09:00:21.827 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T09:00:22.033 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T09:00:22.042 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T09:00:22.043 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T09:00:22.320 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T09:00:22.330 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T09:00:23.331 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T09:00:23.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1457: test_get_osd_id_used_by_pgs: wait_for_clean 2026-02-18T09:00:23.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T09:00:23.611 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T09:00:23.612 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T09:00:23.612 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T09:00:23.613 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:23.613 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:23.613 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:23.613 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:23.613 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:23.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T09:00:23.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T09:00:23.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T09:00:23.710 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T09:00:23.710 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:00:23.710 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:00:23.921 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:00:23.921 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:00:23.922 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:00:23.922 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:00:24.002 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-18T09:00:24.002 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-18T09:00:24.002 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-18T09:00:24.002 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:00:24.002 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-18T09:00:24.002 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:00:24.003 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:00:24.003 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-18T09:00:24.004 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:00:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-18T09:00:24.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-18T09:00:24.005 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836483 2026-02-18T09:00:24.006 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:00:24.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-18T09:00:24.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:00:25.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:00:25.229 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:00:25.447 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-18T09:00:25.448 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T09:00:25.448 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:25.448 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:25.739 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T09:00:25.739 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T09:00:25.739 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T09:00:25.739 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:00:25.739 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:00:25.739 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:00:25.740 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:00:25.740 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:00:25.949 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T09:00:25.949 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T09:00:25.950 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:25.950 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:26.234 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T09:00:26.234 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T09:00:26.234 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T09:00:26.235 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: get_osd_id_used_by_pgs 2026-02-18T09:00:26.235 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: uniq 2026-02-18T09:00:26.235 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: ceph --format json pg dump pgs 2026-02-18T09:00:26.235 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: jq '.pg_stats | .[] | .up[], .acting[]' 2026-02-18T09:00:26.236 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: sort 2026-02-18T09:00:26.441 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: local osd_ids=0 2026-02-18T09:00:26.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1459: test_get_osd_id_used_by_pgs: test 0 = 0 2026-02-18T09:00:26.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1460: test_get_osd_id_used_by_pgs: teardown td/ceph-helpers 2026-02-18T09:00:26.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:26.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:26.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:26.442 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:26.442 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:26.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:26.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:26.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:26.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:26.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:26.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:26.551 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:26.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:26.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:26.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:26.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:26.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:26.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:26.554 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:26.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:26.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:26.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:26.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:26.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:26.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:26.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:26.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:26.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:26.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_osds td/ceph-helpers 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: test_get_osds: local dir=td/ceph-helpers 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1024: test_get_osds: setup td/ceph-helpers 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:26.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:26.566 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:26.566 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:26.567 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:26.567 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:26.568 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:26.568 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:26.569 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:26.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:26.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:26.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:26.571 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:26.571 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:26.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:26.572 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:26.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:26.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:26.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:26.573 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:26.575 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:26.575 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:26.575 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:00:26.576 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:00:26.576 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:26.576 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:26.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:00:26.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:00:26.578 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:00:26.578 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:00:26.578 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:00:26.578 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: test_get_osds: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-18T09:00:26.578 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:00:26.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:00:26.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:00:26.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:00:26.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:00:26.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-18T09:00:26.600 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:00:26.600 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:26.600 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:26.601 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:26.601 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:26.601 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:26.601 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:26.601 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-18T09:00:26.629 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:00:26.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:00:26.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:26.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:26.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:00:26.630 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:00:26.630 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:26.630 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:26.630 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:26.631 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:26.631 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:26.631 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:26.631 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:26.631 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:26.631 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:00:26.685 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:00:26.685 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:26.685 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:26.686 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:00:26.686 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:00:26.686 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:26.686 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:26.686 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:26.686 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:26.686 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:26.686 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:26.687 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:26.687 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:26.687 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:00:26.740 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1026: test_get_osds: run_mgr td/ceph-helpers x 2026-02-18T09:00:26.740 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:00:26.740 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:00:26.740 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:00:26.740 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:00:26.740 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:00:26.740 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:00:26.850 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:00:26.850 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:26.850 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:26.851 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:26.851 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:26.851 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:26.851 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:26.851 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:26.852 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:26.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: test_get_osds: run_osd td/ceph-helpers 0 2026-02-18T09:00:26.871 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:26.872 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:26.873 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:26.873 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:26.873 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:26.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:26.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cd27c115-9daa-4ce7-a3a0-fbb1284b9162 2026-02-18T09:00:26.874 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 cd27c115-9daa-4ce7-a3a0-fbb1284b9162 2026-02-18T09:00:26.875 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 cd27c115-9daa-4ce7-a3a0-fbb1284b9162' 2026-02-18T09:00:26.875 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:00:26.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCqf5Vp9RsNNBAAv7ZiNRcPSQ/7vEBW25IZmA== 2026-02-18T09:00:26.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCqf5Vp9RsNNBAAv7ZiNRcPSQ/7vEBW25IZmA=="}' 2026-02-18T09:00:26.885 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cd27c115-9daa-4ce7-a3a0-fbb1284b9162 -i td/ceph-helpers/0/new.json 2026-02-18T09:00:26.997 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:27.005 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:00:27.006 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCqf5Vp9RsNNBAAv7ZiNRcPSQ/7vEBW25IZmA== --osd-uuid cd27c115-9daa-4ce7-a3a0-fbb1284b9162 2026-02-18T09:00:27.022 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:27.009+0000 7f3c188d9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:27.023 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:27.009+0000 7f3c188d9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:27.024 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:27.009+0000 7f3c188d9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:27.025 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:27.009+0000 7f3c188d9a40 -1 bdev(0x557823e8c400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:00:27.025 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:27.009+0000 7f3c188d9a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:00:27.766 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:00:27.766 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:00:27.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:00:27.767 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:00:27.767 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:00:27.889 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:00:27.890 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:00:27.890 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:00:27.890 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:00:27.890 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:00:27.890 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:00:27.915 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:27.901+0000 7f277dcc9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:27.917 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:27.905+0000 7f277dcc9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:27.919 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:27.905+0000 7f277dcc9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:27.999 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:00:28.000 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:00:28.000 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:00:28.000 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:00:28.000 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:00:28.000 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:28.000 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:28.001 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:00:28.001 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:28.001 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:28.106 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:28.359 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:28.345+0000 7f277dcc9a40 -1 Falling back to public interface 2026-02-18T09:00:29.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:29.108 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:29.108 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:00:29.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:00:29.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:29.109 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:29.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:29.266 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:29.253+0000 7f277dcc9a40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:00:30.220 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:30.220 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:30.221 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:00:30.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:00:30.222 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:30.222 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:30.431 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4034425139,v1:127.0.0.1:6801/4034425139] [v2:127.0.0.1:6802/4034425139,v1:127.0.0.1:6803/4034425139] exists,up cd27c115-9daa-4ce7-a3a0-fbb1284b9162 2026-02-18T09:00:30.435 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:00:30.435 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:00:30.435 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:00:30.435 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1028: test_get_osds: run_osd td/ceph-helpers 1 2026-02-18T09:00:30.435 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:00:30.435 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:00:30.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-18T09:00:30.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:00:30.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-18T09:00:30.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:00:30.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:00:30.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:00:30.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:00:30.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-18T09:00:30.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-18T09:00:30.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:00:30.436 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:00:30.437 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:00:30.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:00:30.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:00:30.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:00:30.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:00:30.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:00:30.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:00:30.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:00:30.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:00:30.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-18T09:00:30.438 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:00:30.438 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4a21a525-cb87-4fe4-a5eb-2b7b5659e1a2 2026-02-18T09:00:30.439 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 4a21a525-cb87-4fe4-a5eb-2b7b5659e1a2' 2026-02-18T09:00:30.439 INFO:tasks.workunit.client.0.trial118.stdout:add osd1 4a21a525-cb87-4fe4-a5eb-2b7b5659e1a2 2026-02-18T09:00:30.439 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:00:30.447 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCuf5VpPib5GRAAz/4G7oAV5U8HYFJn3UZAog== 2026-02-18T09:00:30.447 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCuf5VpPib5GRAAz/4G7oAV5U8HYFJn3UZAog=="}' 2026-02-18T09:00:30.448 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4a21a525-cb87-4fe4-a5eb-2b7b5659e1a2 -i td/ceph-helpers/1/new.json 2026-02-18T09:00:30.667 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:00:30.676 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-18T09:00:30.677 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCuf5VpPib5GRAAz/4G7oAV5U8HYFJn3UZAog== --osd-uuid 4a21a525-cb87-4fe4-a5eb-2b7b5659e1a2 2026-02-18T09:00:30.694 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:30.681+0000 7fa930d13a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:30.695 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:30.681+0000 7fa930d13a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:30.696 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:30.681+0000 7fa930d13a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:30.697 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:30.681+0000 7fa930d13a40 -1 bdev(0x55c100dda400 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-18T09:00:30.697 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:30.681+0000 7fa930d13a40 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-18T09:00:31.356 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-18T09:00:31.357 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:00:31.357 INFO:tasks.workunit.client.0.trial118.stdout:adding osd1 key to auth repository 2026-02-18T09:00:31.358 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-18T09:00:31.358 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:00:31.645 INFO:tasks.workunit.client.0.trial118.stdout:start osd.1 2026-02-18T09:00:31.645 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-18T09:00:31.645 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:00:31.645 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:00:31.645 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:00:31.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:00:31.670 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:31.653+0000 7fa32216ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:31.671 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:31.657+0000 7fa32216ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:31.672 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:31.657+0000 7fa32216ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:31.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-18T09:00:31.862 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:00:31.862 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-18T09:00:31.862 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:00:31.862 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:00:31.862 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:31.862 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:31.862 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:00:31.862 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:31.862 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T09:00:31.999 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:31.985+0000 7fa32216ea40 -1 Falling back to public interface 2026-02-18T09:00:32.078 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:32.699 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:32.685+0000 7fa32216ea40 -1 osd.1 0 log_to_monitors true 2026-02-18T09:00:33.079 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:33.079 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:33.079 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:00:33.080 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:00:33.080 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:33.080 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T09:00:33.292 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:34.294 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:34.294 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:34.294 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:00:34.295 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:00:34.295 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:34.295 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-18T09:00:34.509 INFO:tasks.workunit.client.0.trial118.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4035533026,v1:127.0.0.1:6811/4035533026] [v2:127.0.0.1:6812/4035533026,v1:127.0.0.1:6813/4035533026] exists,up 4a21a525-cb87-4fe4-a5eb-2b7b5659e1a2 2026-02-18T09:00:34.510 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:00:34.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:00:34.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:00:34.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1029: test_get_osds: create_rbd_pool 2026-02-18T09:00:34.511 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T09:00:34.720 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T09:00:34.731 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T09:00:34.731 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T09:00:34.952 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T09:00:34.963 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T09:00:35.965 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T09:00:36.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1030: test_get_osds: wait_for_clean 2026-02-18T09:00:36.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T09:00:36.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T09:00:36.483 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T09:00:36.483 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T09:00:36.483 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:36.484 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:36.484 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:36.484 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:36.484 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:36.583 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T09:00:36.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T09:00:36.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T09:00:36.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T09:00:36.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:00:36.584 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:00:36.799 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-18T09:00:36.799 INFO:tasks.workunit.client.0.trial118.stderr:1' 2026-02-18T09:00:36.799 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:00:36.799 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:00:36.800 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:00:36.878 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-18T09:00:36.878 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-18T09:00:36.878 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-18T09:00:36.878 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:00:36.878 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-18T09:00:36.959 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672963 2026-02-18T09:00:36.959 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672963 2026-02-18T09:00:36.959 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672963' 2026-02-18T09:00:36.959 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:00:36.959 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-18T09:00:36.959 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:00:36.960 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:00:36.960 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-18T09:00:36.961 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:00:36.962 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-18T09:00:36.962 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836484 2026-02-18T09:00:36.963 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-18T09:00:36.963 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:00:37.177 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-18T09:00:37.178 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:00:37.178 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672963 2026-02-18T09:00:37.178 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:00:37.179 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-18T09:00:37.180 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672963 2026-02-18T09:00:37.180 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:00:37.181 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672963 2026-02-18T09:00:37.181 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672963' 2026-02-18T09:00:37.182 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.1 seq 42949672963 2026-02-18T09:00:37.182 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-18T09:00:37.401 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672963 -lt 42949672963 2026-02-18T09:00:37.402 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T09:00:37.402 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:37.402 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:37.687 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T09:00:37.687 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T09:00:37.687 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T09:00:37.688 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:00:37.688 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:00:37.688 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:00:37.688 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:00:37.688 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:00:37.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T09:00:37.895 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T09:00:37.896 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:37.896 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:38.178 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T09:00:38.178 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T09:00:38.178 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T09:00:38.179 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1031: test_get_osds: create_rbd_pool 2026-02-18T09:00:38.179 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T09:00:38.443 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T09:00:38.453 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T09:00:38.453 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T09:00:38.734 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T09:00:38.745 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T09:00:39.746 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T09:00:40.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: get_osds rbd GROUP 2026-02-18T09:00:40.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=rbd 2026-02-18T09:00:40.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=GROUP 2026-02-18T09:00:40.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: grep --quiet '^[0-1] [0-1]$' 2026-02-18T09:00:40.030 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map rbd GROUP 2026-02-18T09:00:40.030 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-18T09:00:40.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-18T09:00:40.247 INFO:tasks.workunit.client.0.trial118.stderr:0' 2026-02-18T09:00:40.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-18T09:00:40.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1033: test_get_osds: teardown td/ceph-helpers 2026-02-18T09:00:40.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:40.247 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:40.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:40.248 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:40.248 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:40.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:40.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:40.248 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:40.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:40.557 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:40.558 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:40.559 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:40.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:40.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:40.561 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:40.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:40.562 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:40.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:40.563 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:40.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:40.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:40.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:40.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:40.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:40.573 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:40.574 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:40.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:40.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:40.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:00:40.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_pg td/ceph-helpers 2026-02-18T09:00:40.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1091: test_get_pg: local dir=td/ceph-helpers 2026-02-18T09:00:40.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: test_get_pg: setup td/ceph-helpers 2026-02-18T09:00:40.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:00:40.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:00:40.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:40.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:40.577 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:40.577 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:40.577 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:40.577 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:40.577 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:40.577 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:40.577 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:40.578 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:40.578 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:40.579 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:40.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:40.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:40.580 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:40.581 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:40.581 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:40.582 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:40.582 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:40.583 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:40.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:40.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:40.585 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:40.586 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:40.586 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:40.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:40.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:40.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:40.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:00:40.588 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:00:40.588 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:40.588 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:40.588 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:00:40.590 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:00:40.591 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:00:40.591 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:00:40.591 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:00:40.591 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: test_get_pg: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:40.591 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:00:40.591 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:00:40.591 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:00:40.591 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:00:40.592 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:00:40.592 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:40.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:00:40.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:40.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:40.614 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:40.614 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:40.614 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:40.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:40.615 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:40.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:00:40.643 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:00:40.643 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:40.643 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:40.643 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:00:40.644 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:00:40.645 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:40.645 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:40.645 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:40.645 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:40.645 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:40.645 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:40.645 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:40.645 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:40.645 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:00:40.700 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:00:40.700 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:40.700 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:40.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:00:40.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:00:40.702 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:40.702 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:40.702 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:40.702 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:40.702 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:40.702 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:40.702 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:40.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:40.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:00:40.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1095: test_get_pg: run_mgr td/ceph-helpers x 2026-02-18T09:00:40.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:00:40.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:00:40.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:00:40.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:00:40.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:00:40.758 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:00:40.870 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:00:40.870 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:40.870 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:40.870 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:40.870 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:40.871 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:40.871 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:40.871 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:40.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:40.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: test_get_pg: run_osd td/ceph-helpers 0 2026-02-18T09:00:40.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:00:40.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:00:40.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:00:40.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:00:40.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:00:40.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:00:40.893 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:40.894 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:00:40.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:00:40.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:00:40.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:00:40.896 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:00:40.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fde04f18-cc23-46ae-9660-88bca391c983 2026-02-18T09:00:40.897 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 fde04f18-cc23-46ae-9660-88bca391c983 2026-02-18T09:00:40.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 fde04f18-cc23-46ae-9660-88bca391c983' 2026-02-18T09:00:40.897 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:00:40.907 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC4f5VpyzNbNRAApM7NQ2ttu97wL/cjSR0uqQ== 2026-02-18T09:00:40.907 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC4f5VpyzNbNRAApM7NQ2ttu97wL/cjSR0uqQ=="}' 2026-02-18T09:00:40.907 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fde04f18-cc23-46ae-9660-88bca391c983 -i td/ceph-helpers/0/new.json 2026-02-18T09:00:41.017 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:41.024 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:00:41.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC4f5VpyzNbNRAApM7NQ2ttu97wL/cjSR0uqQ== --osd-uuid fde04f18-cc23-46ae-9660-88bca391c983 2026-02-18T09:00:41.041 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:41.025+0000 7f69d9983a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:41.042 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:41.029+0000 7f69d9983a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:41.043 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:41.029+0000 7f69d9983a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:41.044 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:41.029+0000 7f69d9983a40 -1 bdev(0x55a0f04b2400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:00:41.044 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:41.029+0000 7f69d9983a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:00:41.781 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:00:41.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:00:41.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:00:41.783 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:00:41.783 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:00:41.907 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:00:41.907 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:00:41.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:00:41.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:00:41.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:00:41.909 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:00:41.932 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:41.917+0000 7f4900bd9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:41.934 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:41.921+0000 7f4900bd9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:41.935 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:41.921+0000 7f4900bd9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:42.018 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:00:42.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:00:42.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:00:42.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:00:42.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:00:42.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:42.019 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:00:42.019 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:42.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:42.020 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:42.128 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:42.339 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:42.325+0000 7f4900bd9a40 -1 Falling back to public interface 2026-02-18T09:00:43.130 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:43.130 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:43.130 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:00:43.131 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:00:43.131 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:43.131 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:43.239 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:43.283 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:43.269+0000 7f4900bd9a40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:00:44.241 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:44.241 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:44.241 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:00:44.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:00:44.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:44.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:44.458 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/170293106,v1:127.0.0.1:6801/170293106] [v2:127.0.0.1:6802/170293106,v1:127.0.0.1:6803/170293106] exists,up fde04f18-cc23-46ae-9660-88bca391c983 2026-02-18T09:00:44.458 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:00:44.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:00:44.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:00:44.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1097: test_get_pg: create_rbd_pool 2026-02-18T09:00:44.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T09:00:44.669 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T09:00:44.678 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T09:00:44.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T09:00:44.959 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T09:00:44.969 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T09:00:45.970 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T09:00:46.255 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1098: test_get_pg: wait_for_clean 2026-02-18T09:00:46.255 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T09:00:46.255 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T09:00:46.255 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T09:00:46.255 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T09:00:46.256 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:46.256 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:46.256 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:46.256 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:46.257 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:46.344 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T09:00:46.344 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T09:00:46.344 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T09:00:46.344 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T09:00:46.344 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:00:46.345 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:00:46.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:00:46.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:00:46.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:00:46.560 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:00:46.641 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-18T09:00:46.641 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-18T09:00:46.641 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-18T09:00:46.641 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:00:46.641 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-18T09:00:46.642 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:00:46.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:00:46.643 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-18T09:00:46.643 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:00:46.644 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-18T09:00:46.644 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-18T09:00:46.644 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836484 2026-02-18T09:00:46.645 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:00:46.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-18T09:00:46.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:00:47.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:00:47.866 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:00:48.083 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-18T09:00:48.084 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T09:00:48.084 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:48.084 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:48.370 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T09:00:48.370 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T09:00:48.370 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T09:00:48.371 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:00:48.371 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:00:48.371 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:00:48.371 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:00:48.371 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:00:48.590 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T09:00:48.591 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T09:00:48.591 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:48.591 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:48.883 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T09:00:48.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T09:00:48.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T09:00:48.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: get_pg rbd GROUP 2026-02-18T09:00:48.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=rbd 2026-02-18T09:00:48.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=GROUP 2026-02-18T09:00:48.884 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: grep --quiet '^[0-9]\.[0-9a-f][0-9a-f]*$' 2026-02-18T09:00:48.885 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map rbd GROUP 2026-02-18T09:00:48.885 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-18T09:00:49.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1100: test_get_pg: teardown td/ceph-helpers 2026-02-18T09:00:49.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:49.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:49.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:49.099 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:49.099 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:49.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:49.099 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:49.100 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:49.205 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:49.206 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:49.206 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:49.207 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:49.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:49.208 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:49.209 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:49.209 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:49.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:49.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:49.210 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:49.211 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:49.212 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:49.213 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:49.218 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:49.218 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:49.218 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:49.218 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:49.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:49.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:49.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:00:49.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_primary td/ceph-helpers 2026-02-18T09:00:49.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: test_get_primary: local dir=td/ceph-helpers 2026-02-18T09:00:49.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: test_get_primary: setup td/ceph-helpers 2026-02-18T09:00:49.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:00:49.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:00:49.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:49.221 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:49.222 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:49.222 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:49.222 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:49.222 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:49.222 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:49.222 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:49.222 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:49.223 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:49.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:49.224 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:49.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:49.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:49.225 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:49.225 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:49.225 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:49.226 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:49.226 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:49.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:49.227 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:49.228 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:49.229 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:49.229 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:49.229 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:49.229 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:49.230 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:49.230 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:49.230 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:00:49.232 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:00:49.232 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:49.232 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:49.232 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:00:49.233 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:00:49.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:00:49.235 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:00:49.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:00:49.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: test_get_primary: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:49.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:00:49.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:00:49.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:00:49.236 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:00:49.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:00:49.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:49.258 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:00:49.258 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:49.258 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:49.259 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:49.259 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:49.259 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:49.259 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:49.259 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:49.288 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:00:49.289 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:00:49.289 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:49.289 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:49.290 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:00:49.290 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:00:49.291 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:49.292 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:49.292 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:49.292 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:49.292 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:49.292 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:49.292 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:49.292 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:49.292 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:00:49.348 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:00:49.348 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:49.349 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:49.349 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:00:49.349 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:00:49.349 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:49.349 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:49.349 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:49.350 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:49.350 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:49.350 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:49.350 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:49.350 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:49.351 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:00:49.404 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: test_get_primary: local osd=0 2026-02-18T09:00:49.404 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: test_get_primary: run_mgr td/ceph-helpers x 2026-02-18T09:00:49.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:00:49.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:00:49.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:00:49.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:00:49.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:00:49.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:00:49.518 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:00:49.519 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:49.519 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:49.519 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:49.519 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:49.519 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:49.519 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:49.520 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:49.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:49.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1201: test_get_primary: run_osd td/ceph-helpers 0 2026-02-18T09:00:49.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:00:49.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:00:49.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:00:49.541 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:00:49.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:00:49.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:00:49.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:00:49.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:00:49.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:00:49.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:00:49.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:00:49.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:00:49.544 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:00:49.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7b3b53a7-2f93-487e-9c64-0cde6e00e527 2026-02-18T09:00:49.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7b3b53a7-2f93-487e-9c64-0cde6e00e527' 2026-02-18T09:00:49.544 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 7b3b53a7-2f93-487e-9c64-0cde6e00e527 2026-02-18T09:00:49.545 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:00:49.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDBf5VpkQJgIBAAF6ak/BYld7TQZKuXfGfHWA== 2026-02-18T09:00:49.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDBf5VpkQJgIBAAF6ak/BYld7TQZKuXfGfHWA=="}' 2026-02-18T09:00:49.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7b3b53a7-2f93-487e-9c64-0cde6e00e527 -i td/ceph-helpers/0/new.json 2026-02-18T09:00:49.667 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:49.675 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:00:49.676 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDBf5VpkQJgIBAAF6ak/BYld7TQZKuXfGfHWA== --osd-uuid 7b3b53a7-2f93-487e-9c64-0cde6e00e527 2026-02-18T09:00:49.692 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:49.676+0000 7f01a6434a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:49.694 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:49.680+0000 7f01a6434a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:49.696 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:49.680+0000 7f01a6434a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:49.696 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:49.680+0000 7f01a6434a40 -1 bdev(0x55b510564400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:00:49.696 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:49.680+0000 7f01a6434a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:00:50.401 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:00:50.401 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:00:50.402 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:00:50.403 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:00:50.403 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:00:50.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:00:50.529 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:00:50.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:00:50.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:00:50.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:00:50.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:00:50.555 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:50.540+0000 7f52936ada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:50.557 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:50.540+0000 7f52936ada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:50.558 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:50.544+0000 7f52936ada40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:50.641 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:00:50.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:00:50.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:00:50.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:00:50.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:00:50.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:50.642 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:50.643 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:00:50.643 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:50.643 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:50.753 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:51.120 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:51.104+0000 7f52936ada40 -1 Falling back to public interface 2026-02-18T09:00:51.754 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:51.754 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:51.755 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:00:51.755 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:00:51.755 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:51.756 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:51.862 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:00:52.071 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:52.056+0000 7f52936ada40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:00:52.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:00:52.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:00:52.864 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:00:52.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:00:52.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:00:52.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:00:53.080 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4203390422,v1:127.0.0.1:6801/4203390422] [v2:127.0.0.1:6802/4203390422,v1:127.0.0.1:6803/4203390422] exists,up 7b3b53a7-2f93-487e-9c64-0cde6e00e527 2026-02-18T09:00:53.080 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:00:53.081 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:00:53.081 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:00:53.081 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: test_get_primary: create_rbd_pool 2026-02-18T09:00:53.081 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T09:00:53.284 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T09:00:53.293 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T09:00:53.293 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T09:00:53.524 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T09:00:53.535 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T09:00:54.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T09:00:54.817 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1203: test_get_primary: wait_for_clean 2026-02-18T09:00:54.817 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T09:00:54.817 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T09:00:54.817 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T09:00:54.817 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T09:00:54.818 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:54.818 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:54.818 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:54.818 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:54.818 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:54.915 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T09:00:54.915 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T09:00:54.915 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T09:00:54.915 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T09:00:54.915 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:00:54.915 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:00:55.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:00:55.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:00:55.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:00:55.132 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:00:55.213 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-18T09:00:55.213 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-18T09:00:55.213 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-18T09:00:55.213 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:00:55.213 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-18T09:00:55.213 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:00:55.215 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:00:55.215 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-18T09:00:55.215 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:00:55.216 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-18T09:00:55.217 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836483 2026-02-18T09:00:55.217 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-18T09:00:55.217 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:00:55.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-18T09:00:55.442 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:00:56.443 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:00:56.443 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:00:56.660 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-18T09:00:56.661 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T09:00:56.661 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:56.661 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:56.950 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T09:00:56.950 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T09:00:56.950 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T09:00:56.950 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:00:56.950 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:00:56.950 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:00:56.950 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:00:56.950 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:00:57.165 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T09:00:57.165 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T09:00:57.165 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:00:57.166 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:00:57.449 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T09:00:57.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T09:00:57.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T09:00:57.450 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: get_primary rbd GROUP 2026-02-18T09:00:57.450 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-18T09:00:57.450 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-18T09:00:57.450 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-18T09:00:57.450 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-18T09:00:57.671 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: test 0 = 0 2026-02-18T09:00:57.671 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1205: test_get_primary: teardown td/ceph-helpers 2026-02-18T09:00:57.671 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:57.671 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:57.671 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:57.671 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:57.671 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:57.671 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:57.671 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:57.671 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:57.779 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:57.780 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:57.781 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:57.781 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:57.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:57.782 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:57.783 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:57.785 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:57.785 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:57.785 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:57.785 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:57.786 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:57.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:57.787 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:57.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:57.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:57.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:57.793 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:57.794 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:57.795 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:57.795 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:00:57.795 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_timeout_delays td/ceph-helpers 2026-02-18T09:00:57.795 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: get_timeout_delays 1 2026-02-18T09:00:57.796 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:57.796 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:57.796 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:57.796 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:57.796 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:57.808 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: test '1 ' = '1 ' 2026-02-18T09:00:57.809 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: get_timeout_delays 5 2026-02-18T09:00:57.809 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:57.810 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:57.810 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:57.810 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:57.810 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:57.833 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: test '1 2 2 ' = '1 2 2 ' 2026-02-18T09:00:57.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: get_timeout_delays 6 2026-02-18T09:00:57.835 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:57.835 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:57.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:57.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:57.835 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:57.859 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: test '1 2 3 ' = '1 2 3 ' 2026-02-18T09:00:57.859 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: get_timeout_delays 7 2026-02-18T09:00:57.860 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:57.860 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:57.861 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:57.861 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:57.861 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:57.891 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: test '1 2 4 ' = '1 2 4 ' 2026-02-18T09:00:57.892 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: get_timeout_delays 8 2026-02-18T09:00:57.892 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:57.893 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:57.893 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:57.893 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:57.893 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:57.925 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: test '1 2 4 1 ' = '1 2 4 1 ' 2026-02-18T09:00:57.926 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: get_timeout_delays 1 .1 2026-02-18T09:00:57.926 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:57.927 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:57.927 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:57.927 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:57.927 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:57.959 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: test '0.1 0.2 0.4 0.3 ' = '0.1 0.2 0.4 0.3 ' 2026-02-18T09:00:57.960 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: get_timeout_delays 1.5 .1 2026-02-18T09:00:57.960 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:57.961 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:57.961 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:57.961 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:57.961 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:57.999 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 ' = '0.1 0.2 0.4 0.8 ' 2026-02-18T09:00:58.000 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: get_timeout_delays 5 .1 2026-02-18T09:00:58.001 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:58.001 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:58.001 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:58.001 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:58.001 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:58.051 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 1.9 ' = '0.1 0.2 0.4 0.8 1.6 1.9 ' 2026-02-18T09:00:58.052 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: get_timeout_delays 6 .1 2026-02-18T09:00:58.052 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:58.052 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:58.052 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:58.052 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:58.053 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:58.101 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 2.9 ' = '0.1 0.2 0.4 0.8 1.6 2.9 ' 2026-02-18T09:00:58.102 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: get_timeout_delays 6.3 .1 2026-02-18T09:00:58.102 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:58.103 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:58.103 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:58.103 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:58.103 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:58.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 ' = '0.1 0.2 0.4 0.8 1.6 3.2 ' 2026-02-18T09:00:58.153 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: get_timeout_delays 20 .1 2026-02-18T09:00:58.154 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:58.154 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:58.154 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:58.154 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:58.154 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:58.222 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' 2026-02-18T09:00:58.222 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: get_timeout_delays 300 .1 0 2026-02-18T09:00:58.223 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:58.223 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:58.223 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:58.224 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:58.224 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:58.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' 2026-02-18T09:00:58.326 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: get_timeout_delays 300 .1 10 2026-02-18T09:00:58.326 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:00:58.326 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:00:58.326 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:00:58.326 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:00:58.326 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:00:58.637 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' 2026-02-18T09:00:58.637 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:00:58.637 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_is_clean td/ceph-helpers 2026-02-18T09:00:58.637 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1552: test_is_clean: local dir=td/ceph-helpers 2026-02-18T09:00:58.637 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1554: test_is_clean: setup td/ceph-helpers 2026-02-18T09:00:58.637 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:00:58.638 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:00:58.638 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:00:58.638 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:00:58.638 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:00:58.638 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:00:58.638 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:00:58.638 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:00:58.638 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:00:58.638 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:00:58.639 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:00:58.639 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:00:58.639 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:00:58.640 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:00:58.641 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:00:58.641 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:00:58.641 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:00:58.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:58.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:00:58.642 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:00:58.642 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:00:58.643 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:00:58.644 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:00:58.644 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:00:58.645 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:00:58.645 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:58.646 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:58.646 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:00:58.647 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:00:58.647 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:00:58.647 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:00:58.648 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:00:58.648 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:58.648 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:58.648 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:00:58.649 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:00:58.651 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:00:58.651 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:00:58.651 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:00:58.651 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1555: test_is_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:58.651 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:00:58.651 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:00:58.651 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:00:58.651 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:00:58.651 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:00:58.651 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:58.673 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:00:58.673 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:58.673 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:58.673 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:58.673 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:58.673 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:58.674 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:58.674 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:00:58.700 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:00:58.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:00:58.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:58.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:58.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:00:58.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:00:58.703 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:58.703 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:58.703 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:58.703 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:58.703 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:58.703 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:58.703 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:58.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:58.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:00:58.757 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:00:58.757 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:00:58.758 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:00:58.758 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:00:58.758 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:00:58.759 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:00:58.759 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:00:58.759 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:00:58.759 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:00:58.759 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:58.759 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:58.759 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:00:58.760 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:00:58.760 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:00:58.813 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1556: test_is_clean: run_mgr td/ceph-helpers x 2026-02-18T09:00:58.813 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:00:58.813 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:00:58.813 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:00:58.814 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:00:58.814 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:00:58.814 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:00:58.923 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:00:58.923 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:58.923 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:58.924 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:58.924 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:58.924 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:58.924 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:58.924 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:58.925 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:00:58.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1557: test_is_clean: run_osd td/ceph-helpers 0 2026-02-18T09:00:58.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:00:58.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:00:58.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:00:58.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:00:58.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:00:58.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:00:58.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:00:58.947 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:00:58.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:00:58.949 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:00:58.949 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:00:58.949 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f868a64d-8566-4ddc-9de7-6dd26d7c7e7b 2026-02-18T09:00:58.949 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f868a64d-8566-4ddc-9de7-6dd26d7c7e7b' 2026-02-18T09:00:58.950 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 f868a64d-8566-4ddc-9de7-6dd26d7c7e7b 2026-02-18T09:00:58.950 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:00:58.959 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDKf5Vp0FV6OBAAoyYzdidurZmBQ6CEAvIi6w== 2026-02-18T09:00:58.959 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDKf5Vp0FV6OBAAoyYzdidurZmBQ6CEAvIi6w=="}' 2026-02-18T09:00:58.959 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f868a64d-8566-4ddc-9de7-6dd26d7c7e7b -i td/ceph-helpers/0/new.json 2026-02-18T09:00:59.074 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:00:59.082 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:00:59.083 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDKf5Vp0FV6OBAAoyYzdidurZmBQ6CEAvIi6w== --osd-uuid f868a64d-8566-4ddc-9de7-6dd26d7c7e7b 2026-02-18T09:00:59.098 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:59.080+0000 7ff3df47fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:59.100 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:59.084+0000 7ff3df47fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:59.101 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:59.084+0000 7ff3df47fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:00:59.102 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:59.084+0000 7ff3df47fa40 -1 bdev(0x555b8763c400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:00:59.102 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:59.084+0000 7ff3df47fa40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:00:59.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:00:59.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:00:59.867 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:00:59.867 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:00:59.868 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:00:59.987 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:00:59.988 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:00:59.989 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:00:59.990 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:00:59.990 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:00:59.990 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:01:00.011 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:59.996+0000 7f56b2ee2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:00.013 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:00:59.996+0000 7f56b2ee2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:00.014 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:00.000+0000 7f56b2ee2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:00.096 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:01:00.096 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:01:00.096 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:01:00.096 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:01:00.096 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:01:00.096 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:00.096 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:01:00.097 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:01:00.097 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:00.097 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:00.202 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:01:00.427 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:00.412+0000 7f56b2ee2a40 -1 Falling back to public interface 2026-02-18T09:01:01.174 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:01.156+0000 7f56b2ee2a40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:01:01.203 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:01:01.204 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:01.204 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:01:01.204 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:01:01.205 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:01.205 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:01.311 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:01:02.312 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:01:02.312 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:02.313 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:01:02.313 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:01:02.313 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:02.314 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:02.528 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3374767378,v1:127.0.0.1:6801/3374767378] [v2:127.0.0.1:6802/3374767378,v1:127.0.0.1:6803/3374767378] exists,up f868a64d-8566-4ddc-9de7-6dd26d7c7e7b 2026-02-18T09:01:02.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:01:02.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:01:02.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:01:02.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1558: test_is_clean: create_rbd_pool 2026-02-18T09:01:02.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T09:01:02.732 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T09:01:02.741 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T09:01:02.741 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T09:01:02.973 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T09:01:02.982 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T09:01:03.984 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T09:01:04.307 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1559: test_is_clean: wait_for_clean 2026-02-18T09:01:04.308 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T09:01:04.308 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T09:01:04.308 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T09:01:04.308 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T09:01:04.309 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:01:04.309 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:01:04.309 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:01:04.309 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:01:04.309 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:01:04.402 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T09:01:04.403 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T09:01:04.403 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T09:01:04.403 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T09:01:04.403 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:01:04.403 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:01:04.619 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:01:04.620 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:01:04.620 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:01:04.620 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:01:04.699 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-18T09:01:04.700 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-18T09:01:04.700 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-18T09:01:04.700 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:01:04.700 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-18T09:01:04.700 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:01:04.701 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:01:04.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-18T09:01:04.702 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:01:04.703 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-18T09:01:04.703 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836483 2026-02-18T09:01:04.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-18T09:01:04.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:04.926 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-18T09:01:04.926 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:01:05.927 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:01:05.928 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:06.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-18T09:01:06.143 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T09:01:06.143 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:01:06.143 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:01:06.427 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T09:01:06.427 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T09:01:06.427 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T09:01:06.428 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:01:06.428 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:01:06.428 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:01:06.428 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:01:06.428 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:01:06.635 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T09:01:06.636 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T09:01:06.636 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:01:06.636 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:01:06.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T09:01:06.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T09:01:06.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T09:01:06.916 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1560: test_is_clean: is_clean 2026-02-18T09:01:06.916 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: get_num_pgs 2026-02-18T09:01:06.917 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:01:06.917 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:01:07.196 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: num_pgs=4 2026-02-18T09:01:07.196 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1547: is_clean: test 4 '!=' 0 2026-02-18T09:01:07.196 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: get_num_active_clean 2026-02-18T09:01:07.197 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:01:07.197 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:01:07.197 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:01:07.197 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:01:07.197 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:01:07.404 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: test 4 = 4 2026-02-18T09:01:07.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1561: test_is_clean: teardown td/ceph-helpers 2026-02-18T09:01:07.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:01:07.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:01:07.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:01:07.405 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:07.405 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:07.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:07.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:07.405 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:07.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:07.513 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:01:07.514 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:01:07.514 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:01:07.515 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:01:07.516 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:01:07.516 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:01:07.517 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:07.517 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:01:07.518 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:01:07.518 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:07.519 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:01:07.519 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:01:07.520 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:01:07.525 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:01:07.525 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:07.525 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:07.525 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:01:07.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:01:07.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:01:07.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:01:07.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_kill_daemon td/ceph-helpers 2026-02-18T09:01:07.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:266: test_kill_daemon: local dir=td/ceph-helpers 2026-02-18T09:01:07.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:267: test_kill_daemon: setup td/ceph-helpers 2026-02-18T09:01:07.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:01:07.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:01:07.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:01:07.528 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:01:07.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:01:07.529 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:07.529 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:07.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:07.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:07.529 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:07.530 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:07.530 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:01:07.531 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:01:07.531 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:01:07.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:01:07.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:01:07.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:01:07.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:07.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:01:07.534 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:01:07.534 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:07.535 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:01:07.536 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:01:07.536 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:01:07.537 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:01:07.537 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:07.537 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:07.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:01:07.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:01:07.539 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:01:07.539 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:01:07.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:01:07.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:07.540 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:07.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:01:07.541 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:01:07.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:01:07.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:01:07.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:01:07.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:268: test_kill_daemon: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:07.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:01:07.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:01:07.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:01:07.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:01:07.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:01:07.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:07.563 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:01:07.563 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:07.563 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:07.563 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:07.563 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:07.564 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:07.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:07.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:07.592 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:01:07.593 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:01:07.593 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:01:07.593 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:01:07.593 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:01:07.593 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:01:07.594 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:01:07.594 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:01:07.595 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:01:07.595 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:01:07.595 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:07.595 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:07.595 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:01:07.595 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:01:07.595 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:01:07.650 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:01:07.650 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:01:07.650 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:01:07.650 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:01:07.650 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:01:07.652 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:01:07.652 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:01:07.652 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:01:07.652 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:01:07.652 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:07.652 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:07.652 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:01:07.652 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:01:07.652 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:01:07.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:269: test_kill_daemon: run_mgr td/ceph-helpers x 2026-02-18T09:01:07.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:01:07.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:01:07.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:01:07.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:01:07.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:01:07.707 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:01:07.822 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:01:07.823 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:07.823 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:07.823 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:07.823 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:07.823 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:07.823 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:07.824 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:01:07.824 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:01:07.844 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:270: test_kill_daemon: run_osd td/ceph-helpers 0 2026-02-18T09:01:07.844 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:01:07.844 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:01:07.844 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:01:07.844 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:01:07.844 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:01:07.844 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:07.845 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:01:07.846 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:01:07.847 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:01:07.847 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:01:07.847 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:01:07.847 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:01:07.847 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b09da997-b682-465a-a54a-fa51e8cb9cba 2026-02-18T09:01:07.848 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 b09da997-b682-465a-a54a-fa51e8cb9cba 2026-02-18T09:01:07.848 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b09da997-b682-465a-a54a-fa51e8cb9cba' 2026-02-18T09:01:07.848 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:01:07.856 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDTf5VpostTMhAAaMEj4PBUq8mGCupACIbzOA== 2026-02-18T09:01:07.856 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDTf5VpostTMhAAaMEj4PBUq8mGCupACIbzOA=="}' 2026-02-18T09:01:07.856 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b09da997-b682-465a-a54a-fa51e8cb9cba -i td/ceph-helpers/0/new.json 2026-02-18T09:01:07.966 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:01:07.974 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:01:07.975 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDTf5VpostTMhAAaMEj4PBUq8mGCupACIbzOA== --osd-uuid b09da997-b682-465a-a54a-fa51e8cb9cba 2026-02-18T09:01:07.992 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:07.976+0000 7f2e76bcca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:07.993 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:07.976+0000 7f2e76bcca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:07.994 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:07.980+0000 7f2e76bcca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:07.995 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:07.980+0000 7f2e76bcca40 -1 bdev(0x5625e3e02400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:01:07.995 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:07.980+0000 7f2e76bcca40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:01:08.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:01:08.757 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:01:08.758 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:01:08.759 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:01:08.759 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:01:08.881 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:01:08.882 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:01:08.882 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:01:08.882 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:01:08.882 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:01:08.882 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:01:08.905 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:08.888+0000 7fe83d66aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:08.908 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:08.892+0000 7fe83d66aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:08.909 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:08.892+0000 7fe83d66aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:08.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:01:08.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:01:08.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:01:08.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:01:08.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:01:08.994 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:08.994 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:01:08.995 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:01:08.995 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:08.995 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:09.108 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:01:09.484 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:09.468+0000 7fe83d66aa40 -1 Falling back to public interface 2026-02-18T09:01:10.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:01:10.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:10.110 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:01:10.111 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:01:10.111 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:10.111 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:10.324 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:01:10.512 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:10.496+0000 7fe83d66aa40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:01:11.325 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:01:11.326 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:11.326 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:01:11.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:01:11.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:11.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:11.545 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1494157189,v1:127.0.0.1:6801/1494157189] [v2:127.0.0.1:6802/1494157189,v1:127.0.0.1:6803/1494157189] exists,up b09da997-b682-465a-a54a-fa51e8cb9cba 2026-02-18T09:01:11.546 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:01:11.547 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:01:11.547 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:01:11.547 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:272: test_kill_daemon: name_prefix=osd 2026-02-18T09:01:11.547 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: find td/ceph-helpers 2026-02-18T09:01:11.547 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: grep 'osd[^/]*\.pid' 2026-02-18T09:01:11.548 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-18T09:01:11.548 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:280: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid 0 1 2026-02-18T09:01:11.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-18T09:01:11.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=94612 2026-02-18T09:01:11.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=0 2026-02-18T09:01:11.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local delays=1 2026-02-18T09:01:11.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-18T09:01:11.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-18T09:01:11.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 94612 2026-02-18T09:01:11.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-18T09:01:11.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-18T09:01:11.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 1 2026-02-18T09:01:12.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-18T09:01:12.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 94612 2026-02-18T09:01:12.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-18T09:01:12.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-18T09:01:12.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0 2026-02-18T09:01:12.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 1 2026-02-18T09:01:12.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:284: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid TERM 2026-02-18T09:01:12.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-18T09:01:12.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=94612 2026-02-18T09:01:12.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-18T09:01:12.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-18T09:01:12.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-18T09:01:12.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-18T09:01:12.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 94612 2026-02-18T09:01:12.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-18T09:01:12.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-18T09:01:12.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-18T09:01:12.656 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-18T09:01:12.656 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 94612 2026-02-18T09:01:12.657 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-18T09:01:12.657 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-18T09:01:12.657 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.2 2026-02-18T09:01:12.858 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-18T09:01:12.858 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 94612 2026-02-18T09:01:12.858 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-18T09:01:12.858 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-18T09:01:12.858 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-18T09:01:12.858 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:287: test_kill_daemon: name_prefix=mgr 2026-02-18T09:01:12.859 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: find td/ceph-helpers 2026-02-18T09:01:12.859 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: grep 'mgr[^/]*\.pid' 2026-02-18T09:01:12.859 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-18T09:01:12.859 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:292: test_kill_daemon: kill_daemon td/ceph-helpers/mgr.x.pid TERM 2026-02-18T09:01:12.860 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mgr.x.pid 2026-02-18T09:01:12.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=93756 2026-02-18T09:01:12.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-18T09:01:12.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-18T09:01:12.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-18T09:01:12.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-18T09:01:12.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 93756 2026-02-18T09:01:12.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-18T09:01:12.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-18T09:01:12.861 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-18T09:01:12.962 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-18T09:01:12.962 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 93756 2026-02-18T09:01:12.963 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-18T09:01:12.963 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-18T09:01:12.963 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-18T09:01:12.963 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:295: test_kill_daemon: name_prefix=mon 2026-02-18T09:01:12.963 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: find td/ceph-helpers 2026-02-18T09:01:12.963 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: grep 'mon[^/]*\.pid' 2026-02-18T09:01:12.964 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-18T09:01:12.964 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:300: test_kill_daemon: kill_daemon td/ceph-helpers/mon.a.pid TERM 2026-02-18T09:01:12.965 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mon.a.pid 2026-02-18T09:01:12.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=93671 2026-02-18T09:01:12.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-18T09:01:12.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-18T09:01:12.966 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-18T09:01:12.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-18T09:01:12.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 93671 2026-02-18T09:01:12.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-18T09:01:12.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-18T09:01:12.967 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-18T09:01:13.067 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-18T09:01:13.067 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 93671 2026-02-18T09:01:13.067 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-18T09:01:13.067 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-18T09:01:13.067 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-18T09:01:13.067 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:301: test_kill_daemon: timeout 5 ceph status 2026-02-18T09:01:18.075 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:304: test_kill_daemon: teardown td/ceph-helpers 2026-02-18T09:01:18.075 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:01:18.075 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:01:18.075 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:01:18.076 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:18.076 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:18.076 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:18.076 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:18.077 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:18.081 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:18.081 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:01:18.082 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:01:18.082 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:01:18.083 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:01:18.083 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:01:18.084 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:01:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:01:18.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:01:18.086 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:18.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:01:18.088 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:01:18.088 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:01:18.091 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:01:18.091 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:18.091 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:18.091 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:01:18.093 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_kill_daemons td/ceph-helpers 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:357: test_kill_daemons: local dir=td/ceph-helpers 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:358: test_kill_daemons: setup td/ceph-helpers 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:18.094 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:18.095 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:18.095 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:18.095 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:18.095 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:18.096 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:01:18.097 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:01:18.097 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:01:18.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:01:18.098 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:01:18.099 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:01:18.100 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:18.100 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:01:18.100 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:01:18.101 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:18.101 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:01:18.102 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:01:18.103 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:01:18.104 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:01:18.104 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:18.104 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:18.104 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:01:18.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:01:18.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:01:18.105 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:01:18.107 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:01:18.107 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:18.107 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:18.107 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:01:18.109 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:01:18.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:01:18.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:01:18.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:01:18.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:359: test_kill_daemons: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:18.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:01:18.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:01:18.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:01:18.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:01:18.110 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:01:18.111 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:18.131 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:01:18.131 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:18.132 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:18.132 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:18.132 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:18.132 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:18.132 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:18.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:18.162 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:01:18.163 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:01:18.163 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:01:18.163 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:01:18.163 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:01:18.163 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:01:18.164 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:01:18.165 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:01:18.165 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:01:18.165 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:01:18.165 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:18.165 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:18.165 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:01:18.165 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:01:18.165 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:01:18.221 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:01:18.222 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:01:18.222 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:01:18.222 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:01:18.222 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:01:18.223 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:01:18.223 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:01:18.223 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:01:18.223 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:01:18.223 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:18.224 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:18.224 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:01:18.224 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:01:18.224 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:01:18.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:360: test_kill_daemons: run_mgr td/ceph-helpers x 2026-02-18T09:01:18.278 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:01:18.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:01:18.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:01:18.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:01:18.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:01:18.279 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:01:18.391 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:01:18.392 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:18.392 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:18.392 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:18.392 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:18.392 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:18.392 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:18.393 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:01:18.393 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:01:18.413 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:361: test_kill_daemons: run_osd td/ceph-helpers 0 2026-02-18T09:01:18.413 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:01:18.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:01:18.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:01:18.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:01:18.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:01:18.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:01:18.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:01:18.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:01:18.414 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:18.415 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:18.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:18.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:01:18.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:01:18.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:01:18.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:01:18.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:01:18.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:01:18.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:01:18.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:01:18.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:01:18.416 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:01:18.417 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:01:18.417 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:01:18.417 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:01:18.417 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3992e426-f7c8-475a-bd06-0021f02c0e63 2026-02-18T09:01:18.418 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3992e426-f7c8-475a-bd06-0021f02c0e63' 2026-02-18T09:01:18.418 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 3992e426-f7c8-475a-bd06-0021f02c0e63 2026-02-18T09:01:18.418 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:01:18.425 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDef5Vpy1OfGBAAuAvXpGwtXFqQjSBFSJbazQ== 2026-02-18T09:01:18.425 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDef5Vpy1OfGBAAuAvXpGwtXFqQjSBFSJbazQ=="}' 2026-02-18T09:01:18.425 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3992e426-f7c8-475a-bd06-0021f02c0e63 -i td/ceph-helpers/0/new.json 2026-02-18T09:01:18.534 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:01:18.542 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:01:18.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDef5Vpy1OfGBAAuAvXpGwtXFqQjSBFSJbazQ== --osd-uuid 3992e426-f7c8-475a-bd06-0021f02c0e63 2026-02-18T09:01:18.560 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:18.544+0000 7ff4abe9ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:18.561 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:18.544+0000 7ff4abe9ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:18.562 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:18.544+0000 7ff4abe9ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:18.562 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:18.548+0000 7ff4abe9ca40 -1 bdev(0x55eefd46a400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:01:18.562 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:18.548+0000 7ff4abe9ca40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:01:19.343 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:01:19.343 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:01:19.344 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:01:19.344 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:01:19.344 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:01:19.468 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:01:19.469 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:01:19.469 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:01:19.469 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:01:19.469 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:01:19.469 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:01:19.492 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:19.476+0000 7fd4f4594a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:19.495 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:19.480+0000 7fd4f4594a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:19.496 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:19.480+0000 7fd4f4594a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:19.581 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:01:19.581 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:01:19.581 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:01:19.581 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:01:19.582 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:01:19.582 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:19.582 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:01:19.582 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:01:19.583 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:19.583 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:19.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:01:19.992 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:19.976+0000 7fd4f4594a40 -1 Falling back to public interface 2026-02-18T09:01:20.694 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:01:20.695 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:20.695 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:01:20.696 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:01:20.696 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:20.696 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:20.910 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:01:20.951 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:20.936+0000 7fd4f4594a40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:01:21.911 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:01:21.911 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:21.911 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:01:21.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:01:21.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:21.912 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:22.131 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1152946449,v1:127.0.0.1:6801/1152946449] [v2:127.0.0.1:6802/1152946449,v1:127.0.0.1:6803/1152946449] exists,up 3992e426-f7c8-475a-bd06-0021f02c0e63 2026-02-18T09:01:22.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:01:22.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:01:22.132 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:01:22.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:368: test_kill_daemons: kill_daemons td/ceph-helpers 0 osd 1 2026-02-18T09:01:22.133 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:22.133 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:22.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:22.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:22.133 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:23.140 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 1 2026-02-18T09:01:23.140 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:372: test_kill_daemons: kill_daemons td/ceph-helpers TERM osd 2026-02-18T09:01:23.141 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:23.141 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:23.141 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:23.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:23.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:23.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:23.450 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:376: test_kill_daemons: kill_daemons td/ceph-helpers TERM mgr 2026-02-18T09:01:23.451 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:23.451 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:23.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:23.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:23.451 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:23.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:23.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:380: test_kill_daemons: kill_daemons td/ceph-helpers TERM 2026-02-18T09:01:23.559 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:23.559 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:23.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:23.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:23.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:23.665 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:23.665 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:381: test_kill_daemons: timeout 5 ceph status 2026-02-18T09:01:28.672 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:382: test_kill_daemons: teardown td/ceph-helpers 2026-02-18T09:01:28.672 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:01:28.672 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:01:28.672 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:01:28.672 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:28.672 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:28.673 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:28.673 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:28.673 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:28.678 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:28.678 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:01:28.679 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:01:28.680 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:01:28.681 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:01:28.681 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:01:28.681 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:01:28.682 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:28.683 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:01:28.683 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:01:28.683 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:28.684 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:01:28.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:01:28.685 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:01:28.690 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:01:28.690 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:28.690 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:28.690 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:01:28.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:01:28.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:01:28.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:01:28.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_objectstore_tool td/ceph-helpers 2026-02-18T09:01:28.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: test_objectstore_tool: local dir=td/ceph-helpers 2026-02-18T09:01:28.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: test_objectstore_tool: setup td/ceph-helpers 2026-02-18T09:01:28.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:01:28.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:01:28.693 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:01:28.694 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:01:28.694 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:01:28.694 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:28.694 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:28.694 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:28.694 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:28.694 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:28.695 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:28.695 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:01:28.696 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:01:28.696 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:01:28.697 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:01:28.698 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:01:28.698 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:01:28.699 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:28.699 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:01:28.699 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:01:28.699 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:28.700 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:01:28.701 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:01:28.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:01:28.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:01:28.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:28.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:28.704 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:01:28.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:01:28.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:01:28.705 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:01:28.706 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:01:28.706 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:28.706 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:28.706 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:01:28.708 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:01:28.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:01:28.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:01:28.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:01:28.709 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1301: test_objectstore_tool: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:28.710 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:01:28.710 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:01:28.710 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:01:28.710 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:01:28.710 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:01:28.710 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:28.730 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:01:28.730 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:28.730 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:28.731 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:28.731 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:28.731 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:28.731 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:28.731 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:28.761 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:01:28.762 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:01:28.763 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:01:28.763 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:01:28.763 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:01:28.763 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:01:28.764 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:01:28.764 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:01:28.764 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:01:28.764 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:01:28.764 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:28.764 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:28.765 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:01:28.765 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:01:28.765 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:01:28.818 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:01:28.818 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:01:28.818 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:01:28.819 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:01:28.819 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:01:28.819 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:01:28.819 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:01:28.819 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:01:28.820 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:01:28.820 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:28.820 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:28.820 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:01:28.820 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:01:28.820 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:01:28.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: test_objectstore_tool: local osd=0 2026-02-18T09:01:28.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: test_objectstore_tool: run_mgr td/ceph-helpers x 2026-02-18T09:01:28.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:01:28.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:01:28.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:01:28.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:01:28.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:01:28.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:01:28.983 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:01:28.983 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:28.983 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:28.983 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:28.983 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:28.983 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:28.983 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:28.984 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:01:28.984 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:01:29.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: test_objectstore_tool: run_osd td/ceph-helpers 0 2026-02-18T09:01:29.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:01:29.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:01:29.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:01:29.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:01:29.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:01:29.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:29.009 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:01:29.010 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:01:29.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:01:29.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:01:29.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:01:29.011 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:01:29.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2a3cfe68-0fe5-4bd0-b203-90cafedff9db 2026-02-18T09:01:29.012 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 2a3cfe68-0fe5-4bd0-b203-90cafedff9db 2026-02-18T09:01:29.012 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2a3cfe68-0fe5-4bd0-b203-90cafedff9db' 2026-02-18T09:01:29.012 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:01:29.021 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDpf5VpQweIABAAWgjB8LOsmN8OlVLQnxUIPw== 2026-02-18T09:01:29.021 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDpf5VpQweIABAAWgjB8LOsmN8OlVLQnxUIPw=="}' 2026-02-18T09:01:29.021 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2a3cfe68-0fe5-4bd0-b203-90cafedff9db -i td/ceph-helpers/0/new.json 2026-02-18T09:01:29.134 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:01:29.142 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:01:29.143 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDpf5VpQweIABAAWgjB8LOsmN8OlVLQnxUIPw== --osd-uuid 2a3cfe68-0fe5-4bd0-b203-90cafedff9db 2026-02-18T09:01:29.160 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:29.143+0000 7f61bfed6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:29.161 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:29.143+0000 7f61bfed6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:29.162 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:29.147+0000 7f61bfed6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:29.163 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:29.147+0000 7f61bfed6a40 -1 bdev(0x56089fd32400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:01:29.163 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:29.147+0000 7f61bfed6a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:01:29.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:01:29.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:01:29.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:01:29.947 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:01:29.947 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:01:30.067 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:01:30.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:01:30.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:01:30.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:01:30.068 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:01:30.069 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:01:30.091 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:30.075+0000 7f83bdd37a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:30.095 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:30.079+0000 7f83bdd37a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:30.096 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:30.079+0000 7f83bdd37a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:30.177 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:01:30.177 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:01:30.178 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:01:30.178 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:01:30.178 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:01:30.178 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:30.178 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:01:30.178 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:01:30.179 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:30.179 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:30.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:01:30.636 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:30.619+0000 7f83bdd37a40 -1 Falling back to public interface 2026-02-18T09:01:31.284 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:01:31.285 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:31.285 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:01:31.285 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:01:31.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:31.286 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:31.396 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:01:31.565 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:31.547+0000 7f83bdd37a40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:01:32.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:01:32.398 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:32.398 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:01:32.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:01:32.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:32.399 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:32.613 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/106278001,v1:127.0.0.1:6801/106278001] [v2:127.0.0.1:6802/106278001,v1:127.0.0.1:6803/106278001] exists,up 2a3cfe68-0fe5-4bd0-b203-90cafedff9db 2026-02-18T09:01:32.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:01:32.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:01:32.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:01:32.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: test_objectstore_tool: create_rbd_pool 2026-02-18T09:01:32.614 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T09:01:32.817 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T09:01:32.826 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T09:01:32.826 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T09:01:33.108 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T09:01:33.119 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T09:01:34.120 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T09:01:34.370 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: test_objectstore_tool: wait_for_clean 2026-02-18T09:01:34.371 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T09:01:34.371 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T09:01:34.371 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T09:01:34.371 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T09:01:34.372 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:01:34.372 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:01:34.372 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:01:34.372 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:01:34.372 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:01:34.471 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T09:01:34.472 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T09:01:34.472 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T09:01:34.472 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T09:01:34.472 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:01:34.472 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:01:34.682 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:01:34.682 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:01:34.682 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:01:34.682 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:01:34.764 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-18T09:01:34.764 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-18T09:01:34.765 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-18T09:01:34.765 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:01:34.765 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-18T09:01:34.765 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:01:34.766 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:01:34.767 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-18T09:01:34.767 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:01:34.768 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-18T09:01:34.768 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-18T09:01:34.769 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836483 2026-02-18T09:01:34.769 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:34.991 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-18T09:01:34.991 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:01:35.992 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:01:35.993 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:36.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-18T09:01:36.211 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T09:01:36.211 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:01:36.212 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:01:36.506 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T09:01:36.506 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T09:01:36.506 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T09:01:36.506 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:01:36.506 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:01:36.506 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:01:36.506 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:01:36.507 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:01:36.718 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T09:01:36.718 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T09:01:36.719 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:01:36.719 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:01:37.007 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T09:01:37.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T09:01:37.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T09:01:37.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1307: test_objectstore_tool: rados --pool rbd put GROUP /etc/group 2026-02-18T09:01:37.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1308: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 GROUP get-bytes 2026-02-18T09:01:37.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-18T09:01:37.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-18T09:01:37.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-18T09:01:37.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-18T09:01:37.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 GROUP get-bytes 2026-02-18T09:01:37.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-18T09:01:37.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-18T09:01:37.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-18T09:01:37.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-18T09:01:37.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-18T09:01:37.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1309: test_objectstore_tool: diff - /etc/group 2026-02-18T09:01:37.030 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:37.030 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:37.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:37.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:37.030 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:37.337 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:37.337 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 GROUP get-bytes 2026-02-18T09:01:37.337 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-18T09:01:37.337 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-18T09:01:37.338 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-18T09:01:37.338 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-18T09:01:37.338 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-18T09:01:37.338 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 GROUP get-bytes 2026-02-18T09:01:37.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/ceph-helpers 0 2026-02-18T09:01:37.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-18T09:01:37.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-18T09:01:37.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-18T09:01:37.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-18T09:01:37.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:01:37.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:01:37.504 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:37.505 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:37.506 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:37.507 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:37.507 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:37.507 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:01:37.507 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr:start osd.0 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:01:37.508 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-18T09:01:37.509 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-18T09:01:37.509 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-18T09:01:37.509 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-18T09:01:37.509 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-18T09:01:37.526 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:37.507+0000 7f56b87c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:37.527 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:37.511+0000 7f56b87c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:37.528 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:37.511+0000 7f56b87c6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:37.723 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-18T09:01:37.724 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:01:37.724 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:01:37.724 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:01:37.724 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:01:37.724 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:37.724 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:01:37.724 INFO:tasks.workunit.client.0.trial118.stderr:0 2026-02-18T09:01:37.724 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:37.724 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:37.904 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:37.887+0000 7f56b87c6a40 -1 Falling back to public interface 2026-02-18T09:01:37.946 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:01:38.519 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:38.503+0000 7f56b87c6a40 -1 osd.0 13 log_to_monitors true 2026-02-18T09:01:38.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:01:38.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:38.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:01:38.948 INFO:tasks.workunit.client.0.trial118.stderr:1 2026-02-18T09:01:38.948 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:38.949 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:39.165 INFO:tasks.workunit.client.0.trial118.stderr:osd.0 up in weight 1 up_from 16 up_thru 16 down_at 13 last_clean_interval [5,13) [v2:127.0.0.1:6800/48473246,v1:127.0.0.1:6801/48473246] [v2:127.0.0.1:6802/48473246,v1:127.0.0.1:6803/48473246] exists,up 2a3cfe68-0fe5-4bd0-b203-90cafedff9db 2026-02-18T09:01:39.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:01:39.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:01:39.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:01:39.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-18T09:01:39.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T09:01:39.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T09:01:39.166 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T09:01:39.166 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T09:01:39.166 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:01:39.166 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:01:39.166 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:01:39.167 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:01:39.167 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:01:39.256 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T09:01:39.256 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T09:01:39.256 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T09:01:39.256 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T09:01:39.256 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:01:39.256 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:01:39.469 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:01:39.469 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:01:39.470 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:01:39.470 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:01:39.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=68719476739 2026-02-18T09:01:39.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 68719476739 2026-02-18T09:01:39.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-68719476739' 2026-02-18T09:01:39.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:01:39.551 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-68719476739 2026-02-18T09:01:39.551 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:01:39.551 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:01:39.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-68719476739 2026-02-18T09:01:39.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:01:39.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=68719476739 2026-02-18T09:01:39.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 68719476739' 2026-02-18T09:01:39.553 INFO:tasks.workunit.client.0.trial118.stderr:waiting osd.0 seq 68719476739 2026-02-18T09:01:39.554 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:39.777 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 68719476739 2026-02-18T09:01:39.777 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:01:40.778 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:01:40.778 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:40.995 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 68719476739 2026-02-18T09:01:40.996 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:01:41.997 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-18T09:01:41.997 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:42.210 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 68719476739 -lt 68719476739 2026-02-18T09:01:42.211 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T09:01:42.211 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:01:42.211 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:01:42.494 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T09:01:42.495 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T09:01:42.495 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T09:01:42.495 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:01:42.495 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:01:42.495 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:01:42.495 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:01:42.495 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:01:42.702 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T09:01:42.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T09:01:42.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:01:42.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:01:42.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T09:01:42.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T09:01:42.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T09:01:42.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1310: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-18T09:01:42.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-18T09:01:42.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-18T09:01:42.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-18T09:01:42.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-18T09:01:42.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-18T09:01:42.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-18T09:01:42.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-18T09:01:42.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-18T09:01:42.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-18T09:01:42.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-18T09:01:42.981 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:42.981 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:42.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:42.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:42.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:43.288 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:43.288 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-18T09:01:43.288 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-18T09:01:43.289 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-18T09:01:43.289 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-18T09:01:43.289 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-18T09:01:43.289 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-18T09:01:43.289 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 NOTEXISTS get-bytes 2026-02-18T09:01:43.349 INFO:tasks.workunit.client.0.trial118.stderr:No object id 'NOTEXISTS' found or invalid JSON specified 2026-02-18T09:01:43.455 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-18T09:01:43.456 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: return 1 2026-02-18T09:01:43.456 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: return 1 2026-02-18T09:01:43.456 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1311: test_objectstore_tool: teardown td/ceph-helpers 2026-02-18T09:01:43.456 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:01:43.456 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:01:43.456 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:01:43.456 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:43.456 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:43.456 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:43.456 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:43.457 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:43.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:43.563 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:01:43.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:01:43.564 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:01:43.566 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:01:43.566 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:01:43.566 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:01:43.567 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:43.567 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:01:43.568 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:01:43.568 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:43.569 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:01:43.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:01:43.570 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:01:43.576 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:01:43.576 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:43.576 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:43.576 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:01:43.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:01:43.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:01:43.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:01:43.579 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_pg_schedule_scrub td/ceph-helpers 2026-02-18T09:01:43.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1987: test_pg_schedule_scrub: local dir=td/ceph-helpers 2026-02-18T09:01:43.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1989: test_pg_schedule_scrub: setup td/ceph-helpers 2026-02-18T09:01:43.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:01:43.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:01:43.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:01:43.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:01:43.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:01:43.580 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:43.580 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:43.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:43.580 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:43.581 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:43.581 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:43.581 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:01:43.582 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:01:43.583 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:01:43.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:01:43.584 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:01:43.584 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:01:43.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:43.585 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:01:43.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:01:43.586 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:43.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:01:43.588 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:01:43.588 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:01:43.589 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:01:43.589 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:43.589 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:43.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:01:43.590 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:01:43.590 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:01:43.590 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:01:43.591 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:01:43.592 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:43.592 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:43.592 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:01:43.593 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:01:43.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:01:43.594 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:01:43.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:01:43.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1990: test_pg_schedule_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:43.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:01:43.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:01:43.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:01:43.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:01:43.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:01:43.595 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:43.615 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:01:43.615 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:43.616 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:43.616 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:43.616 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:43.616 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:43.616 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:43.616 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:43.644 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:01:43.646 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:01:43.646 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:01:43.647 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:01:43.647 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:01:43.647 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:01:43.647 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:01:43.647 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:01:43.647 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:01:43.647 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:01:43.647 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:43.647 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:43.647 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:01:43.648 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:01:43.648 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:01:43.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:01:43.703 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:01:43.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:01:43.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:01:43.704 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:01:43.704 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:01:43.704 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:01:43.704 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:01:43.705 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:01:43.705 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:43.705 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:43.705 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:01:43.705 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:01:43.705 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:01:43.759 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1991: test_pg_schedule_scrub: run_mgr td/ceph-helpers x --mgr_stats_period=1 2026-02-18T09:01:43.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:01:43.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:01:43.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:01:43.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:01:43.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:01:43.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:01:43.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:01:43.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:43.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:43.872 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:43.872 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:43.872 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:43.872 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:43.873 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:01:43.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-18T09:01:43.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1992: test_pg_schedule_scrub: run_osd td/ceph-helpers 0 2026-02-18T09:01:43.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:01:43.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:01:43.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:01:43.895 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:01:43.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:01:43.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:01:43.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:01:43.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:01:43.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:01:43.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:01:43.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:01:43.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:01:43.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:01:43.896 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:01:43.896 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:01:43.897 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:01:43.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:01:43.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:01:43.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:01:43.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:01:43.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:01:43.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:01:43.898 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:01:43.898 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ae5b5297-df9e-4921-afbb-1e5479e44cdd 2026-02-18T09:01:43.899 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 ae5b5297-df9e-4921-afbb-1e5479e44cdd 2026-02-18T09:01:43.899 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ae5b5297-df9e-4921-afbb-1e5479e44cdd' 2026-02-18T09:01:43.899 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:01:43.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD3f5VpYUdoNRAAg03QNWXus7K4H1UTMSv/sA== 2026-02-18T09:01:43.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD3f5VpYUdoNRAAg03QNWXus7K4H1UTMSv/sA=="}' 2026-02-18T09:01:43.908 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ae5b5297-df9e-4921-afbb-1e5479e44cdd -i td/ceph-helpers/0/new.json 2026-02-18T09:01:44.019 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:01:44.026 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:01:44.027 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD3f5VpYUdoNRAAg03QNWXus7K4H1UTMSv/sA== --osd-uuid ae5b5297-df9e-4921-afbb-1e5479e44cdd 2026-02-18T09:01:44.043 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:44.027+0000 7fa099279a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:44.045 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:44.027+0000 7fa099279a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:44.046 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:44.027+0000 7fa099279a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:44.046 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:44.031+0000 7fa099279a40 -1 bdev(0x5644ef1ac400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:01:44.046 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:44.031+0000 7fa099279a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:01:44.734 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:01:44.734 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:01:44.735 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:01:44.736 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:01:44.736 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:01:44.861 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:01:44.862 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:01:44.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:01:44.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:01:44.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:01:44.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:01:44.888 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:44.871+0000 7f6ec094da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:44.890 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:44.871+0000 7f6ec094da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:44.891 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:44.875+0000 7f6ec094da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:01:44.974 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:01:44.974 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:01:44.974 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:01:44.974 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:01:44.974 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:01:44.974 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:44.975 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:01:44.975 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:01:44.975 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:44.976 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:45.085 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:01:45.280 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:45.263+0000 7f6ec094da40 -1 Falling back to public interface 2026-02-18T09:01:46.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:01:46.086 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:46.087 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:01:46.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:01:46.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:46.087 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:46.197 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:01:46.244 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:46.227+0000 7f6ec094da40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:01:47.198 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:01:47.198 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:01:47.199 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:01:47.199 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:01:47.200 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:01:47.200 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:01:47.418 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2416735357,v1:127.0.0.1:6801/2416735357] [v2:127.0.0.1:6802/2416735357,v1:127.0.0.1:6803/2416735357] exists,up ae5b5297-df9e-4921-afbb-1e5479e44cdd 2026-02-18T09:01:47.418 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:01:47.419 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:01:47.419 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:01:47.419 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1993: test_pg_schedule_scrub: create_rbd_pool 2026-02-18T09:01:47.419 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T09:01:47.620 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T09:01:47.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T09:01:47.631 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T09:01:47.862 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T09:01:47.872 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T09:01:48.873 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T09:01:49.153 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1994: test_pg_schedule_scrub: wait_for_clean 2026-02-18T09:01:49.154 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T09:01:49.154 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T09:01:49.154 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T09:01:49.154 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T09:01:49.155 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:01:49.155 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:01:49.155 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:01:49.155 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:01:49.155 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:01:49.241 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T09:01:49.241 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T09:01:49.241 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T09:01:49.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T09:01:49.242 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:01:49.242 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:01:49.459 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:01:49.460 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:01:49.460 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:01:49.460 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:01:49.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-18T09:01:49.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-18T09:01:49.543 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485' 2026-02-18T09:01:49.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:01:49.544 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836485 2026-02-18T09:01:49.544 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:01:49.545 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:01:49.546 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-18T09:01:49.546 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:01:49.547 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836485 2026-02-18T09:01:49.547 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836485 2026-02-18T09:01:49.548 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-18T09:01:49.548 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:49.761 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836485 2026-02-18T09:01:49.762 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:01:50.763 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:01:50.763 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:50.984 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-18T09:01:50.985 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T09:01:50.985 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:01:50.985 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:01:51.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T09:01:51.277 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T09:01:51.277 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T09:01:51.277 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:01:51.277 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:01:51.277 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:01:51.277 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:01:51.277 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:01:51.492 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T09:01:51.492 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T09:01:51.493 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:01:51.493 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:01:51.783 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T09:01:51.784 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T09:01:51.784 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T09:01:51.784 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1995: test_pg_schedule_scrub: pg_schedule_scrub 1.0 2026-02-18T09:01:51.784 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1969: pg_schedule_scrub: local pgid=1.0 2026-02-18T09:01:51.784 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1971: pg_schedule_scrub: wait_for_pg_clean 1.0 2026-02-18T09:01:51.784 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-18T09:01:51.784 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-18T09:01:51.784 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:01:51.784 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:01:51.784 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:01:51.784 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:01:51.785 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:01:52.028 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-18T09:01:52.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-18T09:01:52.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-18T09:01:52.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-18T09:01:52.029 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:01:52.029 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:01:52.245 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:01:52.245 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:01:52.245 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:01:52.246 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:01:52.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-18T09:01:52.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-18T09:01:52.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836489' 2026-02-18T09:01:52.327 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:01:52.327 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836489 2026-02-18T09:01:52.328 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:01:52.328 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:01:52.329 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-18T09:01:52.329 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:01:52.331 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836489 2026-02-18T09:01:52.331 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-18T09:01:52.331 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836489 2026-02-18T09:01:52.332 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:52.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-18T09:01:52.544 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:01:53.545 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:01:53.546 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:53.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-18T09:01:53.761 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:01:54.762 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-18T09:01:54.762 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:01:54.979 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836491 -lt 21474836489 2026-02-18T09:01:54.979 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-18T09:01:54.979 INFO:tasks.workunit.client.0.trial118.stdout:#---------- 1.0 loop 0 2026-02-18T09:01:54.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-18T09:01:54.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-18T09:01:54.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-18T09:01:54.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-18T09:01:54.980 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-18T09:01:54.980 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-18T09:01:55.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-18T09:01:55.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-18T09:01:55.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-18T09:01:55.061 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-18T09:01:55.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: get_last_scrub_stamp 1.0 2026-02-18T09:01:55.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-18T09:01:55.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-18T09:01:55.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-18T09:01:55.062 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-18T09:01:55.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: local last_scrub=2026-02-18T09:01:47.795432+0000 2026-02-18T09:01:55.267 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1973: pg_schedule_scrub: ceph tell 1.0 schedule-scrub 2026-02-18T09:01:55.340 INFO:tasks.workunit.client.0.trial118.stdout:{ 2026-02-18T09:01:55.341 INFO:tasks.workunit.client.0.trial118.stdout: "deep": false, 2026-02-18T09:01:55.341 INFO:tasks.workunit.client.0.trial118.stdout: "must": false, 2026-02-18T09:01:55.341 INFO:tasks.workunit.client.0.trial118.stdout: "stamp": "2026-02-15T21:01:55.329204+0000" 2026-02-18T09:01:55.341 INFO:tasks.workunit.client.0.trial118.stdout:} 2026-02-18T09:01:55.349 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1974: pg_schedule_scrub: wait_for_scrub 1.0 2026-02-18T09:01:47.795432+0000 2026-02-18T09:01:55.349 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-18T09:01:55.349 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-18T09:01:47.795432+0000 2026-02-18T09:01:55.349 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-18T09:01:55.349 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-18T09:01:55.349 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-18T09:01:55.349 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-18T09:01:55.349 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-18T09:01:55.349 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-18T09:01:55.350 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-18T09:01:55.350 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-18T09:01:55.565 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-18T09:01:47.795432+0000 '>' 2026-02-18T09:01:47.795432+0000 2026-02-18T09:01:55.566 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-18T09:01:56.567 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-18T09:01:56.567 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-18T09:01:56.568 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-18T09:01:56.568 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-18T09:01:56.568 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-18T09:01:56.568 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-18T09:01:56.568 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-18T09:01:56.790 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-15T21:01:55.329204+0000 '>' 2026-02-18T09:01:47.795432+0000 2026-02-18T09:01:56.791 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-18T09:01:57.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-18T09:01:57.792 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-18T09:01:57.792 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-18T09:01:57.792 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-18T09:01:57.792 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-18T09:01:57.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-18T09:01:57.793 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-18T09:01:58.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-15T21:01:55.329204+0000 '>' 2026-02-18T09:01:47.795432+0000 2026-02-18T09:01:58.008 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-18T09:01:59.009 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-18T09:01:59.009 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-18T09:01:59.010 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-18T09:01:59.010 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-18T09:01:59.010 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-18T09:01:59.010 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-18T09:01:59.010 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-18T09:01:59.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-18T09:01:57.180878+0000 '>' 2026-02-18T09:01:47.795432+0000 2026-02-18T09:01:59.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-18T09:01:59.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1996: test_pg_schedule_scrub: teardown td/ceph-helpers 2026-02-18T09:01:59.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:01:59.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:01:59.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:01:59.223 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:59.223 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:59.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:59.223 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:59.224 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:59.532 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:59.532 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:01:59.533 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:01:59.534 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:01:59.535 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:01:59.535 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:01:59.536 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:01:59.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:59.537 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:01:59.538 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:01:59.538 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:59.539 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:01:59.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:01:59.540 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:01:59.546 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:01:59.546 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:59.546 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:59.546 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:01:59.548 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:01:59.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:01:59.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-18T09:01:59.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_pg_scrub td/ceph-helpers 2026-02-18T09:01:59.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: test_pg_scrub: local dir=td/ceph-helpers 2026-02-18T09:01:59.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: test_pg_scrub: setup td/ceph-helpers 2026-02-18T09:01:59.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-18T09:01:59.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-18T09:01:59.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T09:01:59.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-18T09:01:59.549 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T09:01:59.549 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:01:59.550 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:01:59.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:01:59.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:01:59.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:01:59.550 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:01:59.551 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T09:01:59.552 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T09:01:59.552 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T09:01:59.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T09:01:59.553 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T09:01:59.553 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T09:01:59.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:59.554 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T09:01:59.555 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T09:01:59.555 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T09:01:59.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T09:01:59.557 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-18T09:01:59.557 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T09:01:59.558 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T09:01:59.559 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:59.559 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:59.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T09:01:59.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T09:01:59.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T09:01:59.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-18T09:01:59.561 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-18T09:01:59.561 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:59.561 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:59.561 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.53296 2026-02-18T09:01:59.562 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-18T09:01:59.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-18T09:01:59.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-18T09:01:59.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-18T09:01:59.563 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: test_pg_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:59.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-18T09:01:59.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-18T09:01:59.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-18T09:01:59.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-18T09:01:59.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-18T09:01:59.564 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:59.585 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-18T09:01:59.585 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:59.586 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:59.586 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:59.586 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:59.586 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:59.586 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:59.586 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-18T09:01:59.612 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-18T09:01:59.613 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-18T09:01:59.613 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:01:59.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:01:59.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-18T09:01:59.614 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-18T09:01:59.615 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:01:59.615 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:01:59.615 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:01:59.615 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:01:59.615 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:59.615 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:59.615 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:01:59.616 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:01:59.616 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get fsid 2026-02-18T09:01:59.671 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-18T09:01:59.671 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-18T09:01:59.671 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-18T09:01:59.671 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-18T09:01:59.671 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-18T09:01:59.672 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-18T09:01:59.673 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-18T09:01:59.673 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-18T09:01:59.673 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-18T09:01:59.673 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:59.673 INFO:tasks.workunit.client.0.trial118.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:59.673 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.53296/ceph-mon.a.asok 2026-02-18T09:01:59.673 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-18T09:01:59.673 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.53296/ceph-mon.a.asok config get mon_host 2026-02-18T09:01:59.728 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: test_pg_scrub: run_mgr td/ceph-helpers x 2026-02-18T09:01:59.728 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-18T09:01:59.728 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-18T09:01:59.728 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-18T09:01:59.728 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-18T09:01:59.728 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-18T09:01:59.728 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-18T09:01:59.842 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-18T09:01:59.842 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:59.842 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:59.842 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:59.842 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:59.842 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:59.842 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:59.843 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:01:59.843 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-18T09:01:59.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: test_pg_scrub: run_osd td/ceph-helpers 0 2026-02-18T09:01:59.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-18T09:01:59.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-18T09:01:59.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-18T09:01:59.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-18T09:01:59.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-18T09:01:59.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-18T09:01:59.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-18T09:01:59.863 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T09:01:59.864 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-18T09:01:59.865 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-18T09:01:59.866 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-18T09:01:59.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bfc61504-49ca-4201-b811-3c705879e0eb 2026-02-18T09:01:59.866 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 bfc61504-49ca-4201-b811-3c705879e0eb' 2026-02-18T09:01:59.867 INFO:tasks.workunit.client.0.trial118.stdout:add osd0 bfc61504-49ca-4201-b811-3c705879e0eb 2026-02-18T09:01:59.867 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-18T09:01:59.875 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAHgJVplTZwMxAA9Ee6+SNFvnOfYiKD7wlXOw== 2026-02-18T09:01:59.875 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAHgJVplTZwMxAA9Ee6+SNFvnOfYiKD7wlXOw=="}' 2026-02-18T09:01:59.875 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bfc61504-49ca-4201-b811-3c705879e0eb -i td/ceph-helpers/0/new.json 2026-02-18T09:01:59.984 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:01:59.992 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-18T09:01:59.993 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAHgJVplTZwMxAA9Ee6+SNFvnOfYiKD7wlXOw== --osd-uuid bfc61504-49ca-4201-b811-3c705879e0eb 2026-02-18T09:02:00.009 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:59.995+0000 7fed0e76ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:02:00.010 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:59.995+0000 7fed0e76ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:02:00.011 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:59.999+0000 7fed0e76ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:02:00.011 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:59.999+0000 7fed0e76ca40 -1 bdev(0x5555937c8400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-18T09:02:00.012 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:01:59.999+0000 7fed0e76ca40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-18T09:02:00.698 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-18T09:02:00.698 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-18T09:02:00.699 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-18T09:02:00.700 INFO:tasks.workunit.client.0.trial118.stdout:adding osd0 key to auth repository 2026-02-18T09:02:00.700 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-18T09:02:00.825 INFO:tasks.workunit.client.0.trial118.stdout:start osd.0 2026-02-18T09:02:00.826 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-18T09:02:00.826 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=5732f73f-556d-4f75-b124-5feab175747b --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.53296/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-18T09:02:00.826 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-18T09:02:00.826 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-18T09:02:00.826 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-18T09:02:00.851 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:02:00.839+0000 7f37d3c2aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:02:00.853 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:02:00.839+0000 7f37d3c2aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:02:00.854 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:02:00.843+0000 7f37d3c2aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-18T09:02:00.937 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-18T09:02:00.937 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-18T09:02:00.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-18T09:02:00.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-18T09:02:00.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-18T09:02:00.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:02:00.938 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-18T09:02:00.938 INFO:tasks.workunit.client.0.trial118.stdout:0 2026-02-18T09:02:00.939 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:02:00.939 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:02:01.048 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:02:01.412 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:02:01.399+0000 7f37d3c2aa40 -1 Falling back to public interface 2026-02-18T09:02:02.049 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:02:02.050 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:02:02.050 INFO:tasks.workunit.client.0.trial118.stdout:1 2026-02-18T09:02:02.051 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-18T09:02:02.051 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:02:02.051 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:02:02.161 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-18T09:02:02.372 INFO:tasks.workunit.client.0.trial118.stderr:2026-02-18T09:02:02.359+0000 7f37d3c2aa40 -1 osd.0 0 log_to_monitors true 2026-02-18T09:02:03.163 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-18T09:02:03.163 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-18T09:02:03.163 INFO:tasks.workunit.client.0.trial118.stdout:2 2026-02-18T09:02:03.164 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-18T09:02:03.164 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-18T09:02:03.164 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-18T09:02:03.381 INFO:tasks.workunit.client.0.trial118.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3351208965,v1:127.0.0.1:6801/3351208965] [v2:127.0.0.1:6802/3351208965,v1:127.0.0.1:6803/3351208965] exists,up bfc61504-49ca-4201-b811-3c705879e0eb 2026-02-18T09:02:03.382 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-18T09:02:03.382 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-18T09:02:03.382 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-18T09:02:03.382 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1947: test_pg_scrub: create_rbd_pool 2026-02-18T09:02:03.382 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-18T09:02:03.590 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' does not exist 2026-02-18T09:02:03.599 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-18T09:02:03.600 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-18T09:02:03.879 INFO:tasks.workunit.client.0.trial118.stderr:pool 'rbd' already exists 2026-02-18T09:02:03.888 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-18T09:02:04.890 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-18T09:02:05.168 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1948: test_pg_scrub: wait_for_clean 2026-02-18T09:02:05.169 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-18T09:02:05.169 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-18T09:02:05.169 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-18T09:02:05.169 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-18T09:02:05.170 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:02:05.170 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:02:05.170 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:02:05.170 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:02:05.170 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:02:05.261 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-18T09:02:05.262 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-18T09:02:05.262 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-18T09:02:05.262 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-18T09:02:05.262 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:02:05.262 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:02:05.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:02:05.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:02:05.474 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:02:05.474 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:02:05.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-18T09:02:05.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-18T09:02:05.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-18T09:02:05.556 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:02:05.556 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-18T09:02:05.556 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:02:05.557 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:02:05.558 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-18T09:02:05.558 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:02:05.559 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-18T09:02:05.559 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836483 2026-02-18T09:02:05.560 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-18T09:02:05.560 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:02:05.772 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-18T09:02:05.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:02:06.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:02:06.774 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:02:06.989 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-18T09:02:06.990 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-18T09:02:06.990 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:02:06.990 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:02:07.268 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-18T09:02:07.268 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-18T09:02:07.268 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-18T09:02:07.268 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-18T09:02:07.269 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-18T09:02:07.269 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-18T09:02:07.269 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-18T09:02:07.269 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-18T09:02:07.482 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-18T09:02:07.483 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-18T09:02:07.483 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-18T09:02:07.483 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-18T09:02:07.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-18T09:02:07.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-18T09:02:07.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-18T09:02:07.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1949: test_pg_scrub: pg_scrub 1.0 2026-02-18T09:02:07.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-18T09:02:07.773 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-18T09:02:07.774 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-18T09:02:07.774 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-18T09:02:07.774 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:02:07.774 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:02:07.774 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:02:07.775 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:02:07.775 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:02:08.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-18T09:02:08.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-18T09:02:08.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-18T09:02:08.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-18T09:02:08.011 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-18T09:02:08.011 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:02:08.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:02:08.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:02:08.237 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:02:08.237 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-18T09:02:08.316 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-18T09:02:08.316 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-18T09:02:08.317 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485' 2026-02-18T09:02:08.317 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-18T09:02:08.317 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836485 2026-02-18T09:02:08.317 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-18T09:02:08.317 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-18T09:02:08.318 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-18T09:02:08.318 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-18T09:02:08.320 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836485 2026-02-18T09:02:08.320 INFO:tasks.workunit.client.0.trial118.stdout:waiting osd.0 seq 21474836485 2026-02-18T09:02:08.321 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-18T09:02:08.321 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:02:08.546 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-18T09:02:08.546 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:02:09.547 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-18T09:02:09.548 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:02:09.758 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-18T09:02:09.759 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-18T09:02:10.760 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-18T09:02:10.760 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-18T09:02:10.980 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-18T09:02:10.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-18T09:02:10.981 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-18T09:02:10.981 INFO:tasks.workunit.client.0.trial118.stdout:#---------- 1.0 loop 0 2026-02-18T09:02:10.982 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-18T09:02:10.982 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-18T09:02:10.982 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-18T09:02:10.982 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-18T09:02:10.982 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-18T09:02:11.063 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-18T09:02:11.063 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-18T09:02:11.063 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-18T09:02:11.063 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-18T09:02:11.064 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 1.0 2026-02-18T09:02:11.064 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-18T09:02:11.064 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-18T09:02:11.064 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-18T09:02:11.064 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-18T09:02:11.273 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-18T09:02:03.812964+0000 2026-02-18T09:02:11.273 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 1.0 2026-02-18T09:02:11.479 INFO:tasks.workunit.client.0.trial118.stderr:instructing pg 1.0 on osd.0 to scrub 2026-02-18T09:02:11.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 1.0 2026-02-18T09:02:03.812964+0000 2026-02-18T09:02:11.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-18T09:02:11.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-18T09:02:03.812964+0000 2026-02-18T09:02:11.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-18T09:02:11.489 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-18T09:02:11.490 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-18T09:02:11.490 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-18T09:02:11.490 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-18T09:02:11.490 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-18T09:02:11.490 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-18T09:02:11.490 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-18T09:02:11.698 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-18T09:02:03.812964+0000 '>' 2026-02-18T09:02:03.812964+0000 2026-02-18T09:02:11.699 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-18T09:02:12.700 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-18T09:02:12.700 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-18T09:02:12.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-18T09:02:12.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-18T09:02:12.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-18T09:02:12.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-18T09:02:12.701 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-18T09:02:12.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-18T09:02:03.812964+0000 '>' 2026-02-18T09:02:03.812964+0000 2026-02-18T09:02:12.920 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-18T09:02:13.921 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-18T09:02:13.921 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-18T09:02:13.922 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-18T09:02:13.923 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-18T09:02:13.923 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-18T09:02:13.923 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-18T09:02:13.923 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-18T09:02:14.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-18T09:02:12.378148+0000 '>' 2026-02-18T09:02:03.812964+0000 2026-02-18T09:02:14.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-18T09:02:14.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1950: test_pg_scrub: kill_daemons td/ceph-helpers KILL osd 2026-02-18T09:02:14.138 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T09:02:14.138 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T09:02:14.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T09:02:14.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T09:02:14.138 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T09:02:14.245 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T09:02:14.246 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: WAIT_FOR_CLEAN_TIMEOUT=10 2026-02-18T09:02:14.246 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: TIMEOUT=2 2026-02-18T09:02:14.246 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: pg_scrub 1.0 2026-02-18T09:02:14.246 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-18T09:02:14.246 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-18T09:02:14.246 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-18T09:02:14.246 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 10 1 3 2026-02-18T09:02:14.246 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-18T09:02:14.246 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-18T09:02:14.247 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-18T09:02:14.247 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-18T09:02:14.247 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-18T09:02:14.282 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '1') 2026-02-18T09:02:14.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-18T09:02:14.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-18T09:02:14.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-18T09:02:14.283 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=2 2026-02-18T09:02:14.283 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-18T09:02:14.506 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-18T09:02:14.506 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-18T09:02:14.506 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-18T09:02:14.507 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 2 ceph tell osd.0 flush_pg_stats 2026-02-18T09:02:16.512 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-18T09:02:16.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-18T09:02:16.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-18T09:02:16.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-18T09:02:16.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-18T09:02:16.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-18T09:02:16.513 INFO:tasks.workunit.client.0.trial118.stdout:#---------- 1.0 loop 0 2026-02-18T09:02:16.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-18T09:02:16.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-18T09:02:16.513 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-18T09:02:16.513 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-18T09:02:16.514 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-18T11:58:43.471 DEBUG:teuthology.orchestra.run:got remote process result: 124 2026-02-18T11:58:43.492 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-18T11:58:43.492 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1: is_pg_clean: teardown td/ceph-helpers 1 2026-02-18T11:58:43.492 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-18T11:58:43.492 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=1 2026-02-18T11:58:43.492 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-18T11:58:43.492 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-18T11:58:43.492 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-18T11:58:43.492 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-18T11:58:43.492 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-18T11:58:43.493 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-18T11:58:43.569 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-18T11:58:43.570 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-18T11:58:43.570 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-18T11:58:43.571 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-18T11:58:43.572 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-18T11:58:43.572 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-18T11:58:43.573 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-18T11:58:43.574 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T11:58:43.574 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-18T11:58:43.574 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-18T11:58:43.575 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-18T11:58:43.576 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-18T11:58:43.577 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 1 = 1 ']' 2026-02-18T11:58:43.577 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:181: teardown: '[' -n '' ']' 2026-02-18T11:58:43.577 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:185: teardown: mkdir -p /home/ubuntu/cephtest/archive/log 2026-02-18T11:58:43.579 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:186: teardown: mv td/ceph-helpers/mgr.x.log td/ceph-helpers/mon.a.log td/ceph-helpers/osd.0.log /home/ubuntu/cephtest/archive/log 2026-02-18T11:58:43.580 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-18T11:58:43.585 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-18T11:58:43.585 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-18T11:58:43.585 INFO:tasks.workunit.client.0.trial118.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.53296 2026-02-18T11:58:43.585 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.53296 2026-02-18T11:58:43.586 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-18T11:58:43.587 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-18T11:58:43.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-18T11:58:43.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 0 >= 5 )) 2026-02-18T11:58:43.587 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 1 2026-02-18T11:58:44.588 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-18T11:58:44.588 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-18T11:58:44.588 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 1' 2026-02-18T11:58:44.589 INFO:tasks.workunit.client.0.trial118.stdout:#---------- 1.0 loop 1 2026-02-18T11:58:44.589 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-18T11:58:44.590 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-18T11:58:44.590 INFO:tasks.workunit.client.0.trial118.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-18T11:58:44.590 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-18T11:58:44.590 INFO:tasks.workunit.client.0.trial118.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-18T11:59:43.492 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2026-02-18T12:00:43.493 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2026-02-18T12:00:43.494 INFO:tasks.workunit:Stopping ['misc'] on client.0... 2026-02-18T12:00:43.494 DEBUG:teuthology.orchestra.run.trial118:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-18T12:00:43.818 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa/tasks/workunit.py", line 144, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa/tasks/workunit.py", line 292, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_325850157d309f65985eea9ec9bb9f1d56726fee/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test misc/test-ceph-helpers.sh) on trial118 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh' 2026-02-18T12:00:43.823 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-18T12:00:43.839 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-18T12:00:43.839 DEBUG:teuthology.orchestra.run.trial118:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-18T12:00:43.950 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-18T12:00:43.962 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-18T12:00:43.962 DEBUG:teuthology.orchestra.run.trial118:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-18T12:00:44.069 INFO:teuthology.orchestra.run.trial118.stdout: remote refid st t when poll reach delay offset jitter 2026-02-18T12:00:44.069 INFO:teuthology.orchestra.run.trial118.stdout:============================================================================== 2026-02-18T12:00:44.069 INFO:teuthology.orchestra.run.trial118.stdout:-172-234-37-140. 81.104.22.229 2 u 6 512 377 7.369 -17.191 5.543 2026-02-18T12:00:44.069 INFO:teuthology.orchestra.run.trial118.stdout:*pool-96-231-54- .E810. 1 u 421 512 277 12.319 +0.997 1.855 2026-02-18T12:00:44.069 INFO:teuthology.orchestra.run.trial118.stdout:+nyc2.us.ntp.li 17.253.2.35 2 u 483 512 377 3.184 +1.610 0.368 2026-02-18T12:00:44.070 INFO:teuthology.orchestra.run.trial118.stdout:+198.137.202.32 17.253.16.125 2 u 438 512 377 64.615 +0.846 2.237 2026-02-18T12:00:44.071 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-18T12:00:44.083 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-18T12:00:44.083 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-18T12:00:44.095 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-18T12:00:44.107 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-18T12:00:44.122 INFO:teuthology.task.internal:Duration was 11466.011099 seconds 2026-02-18T12:00:44.122 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-18T12:00:44.136 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-18T12:00:44.136 DEBUG:teuthology.orchestra.run.trial118:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-18T12:00:44.163 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-18T12:00:44.163 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial118.front.sepia.ceph.com 2026-02-18T12:00:44.163 DEBUG:teuthology.orchestra.run.trial118:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-18T12:00:44.211 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-18T12:00:44.211 DEBUG:teuthology.orchestra.run.trial118:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-18T12:00:44.421 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-18T12:00:44.422 DEBUG:teuthology.orchestra.run.trial118:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-18T12:00:44.432 INFO:teuthology.orchestra.run.trial118.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-18T12:00:44.433 INFO:teuthology.orchestra.run.trial118.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-18T12:00:44.433 INFO:teuthology.orchestra.run.trial118.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-18T12:00:44.433 INFO:teuthology.orchestra.run.trial118.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-18T12:00:44.433 INFO:teuthology.orchestra.run.trial118.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-18T12:00:44.482 INFO:teuthology.orchestra.run.trial118.stderr: 81.7% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-18T12:00:44.483 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-18T12:00:44.499 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-18T12:00:44.499 DEBUG:teuthology.orchestra.run.trial118:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-18T12:00:44.533 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-18T12:00:44.549 DEBUG:teuthology.orchestra.run.trial118:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-18T12:00:44.584 INFO:teuthology.orchestra.run.trial118.stdout:kernel.core_pattern = core 2026-02-18T12:00:44.595 DEBUG:teuthology.orchestra.run.trial118:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-18T12:00:44.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-18T12:00:44.640 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-18T12:00:44.657 INFO:teuthology.task.internal:Transferring archived files... 2026-02-18T12:00:44.660 DEBUG:teuthology.misc:Transferring archived files from trial118:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/skanta-2026-02-18_08:32:17-rados-wip-bharath18-testing-2026-02-13-0856-distro-default-trial/55841/remote/trial118 2026-02-18T12:00:44.661 DEBUG:teuthology.orchestra.run.trial118:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-18T12:00:58.299 INFO:teuthology.task.internal:Removing archive directory... 2026-02-18T12:00:58.299 DEBUG:teuthology.orchestra.run.trial118:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-18T12:00:58.393 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-18T12:00:58.403 INFO:teuthology.task.internal:Not uploading archives. 2026-02-18T12:00:58.403 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-18T12:00:58.413 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-18T12:00:58.413 DEBUG:teuthology.orchestra.run.trial118:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-18T12:00:58.417 INFO:teuthology.orchestra.run.trial118.stdout: 1180220 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 18 12:00 /home/ubuntu/cephtest 2026-02-18T12:00:58.417 INFO:teuthology.orchestra.run.trial118.stdout: 1190961 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 18 08:51 /home/ubuntu/cephtest/mnt.0 2026-02-18T12:00:58.417 INFO:teuthology.orchestra.run.trial118.stdout: 1190962 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 18 08:58 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-18T12:00:58.417 INFO:teuthology.orchestra.run.trial118.stdout: 1206874 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 18 08:58 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-18T12:00:58.417 INFO:teuthology.orchestra.run.trial118.stdout: 1206875 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Feb 18 11:58 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td 2026-02-18T12:00:58.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-18T12:00:58.418 INFO:teuthology.orchestra.run.trial118.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-18T12:00:58.418 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial118 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-18T12:00:58.419 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-18T12:00:58.431 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-18T12:00:58.441 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial118 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-18T12:00:58.445 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} duration: 11466.011099338531 failure_reason: 'Command failed (workunit test misc/test-ceph-helpers.sh) on trial118 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=325850157d309f65985eea9ec9bb9f1d56726fee TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh''' flavor: default owner: scheduled_skanta@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-18T12:00:58.445 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-18T12:01:00.564 INFO:teuthology.run:FAIL