2026-02-12T19:01:23.211 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-12T19:01:23.227 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T19:01:23.303 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/46611 branch: wip-yuri4-testing-2026-02-11-2126 description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '46611' kernel: branch: distro kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri4-testing-2026-02-11-2126 ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 workunit: branch: wip-yuri4-testing-2026-02-11-2126 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 7172 sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri4-testing-2026-02-11-2126 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 targets: trial010.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE32uu5OE6vz0TKfSCGUkRKKB67/6pyMFH1P0tpl0dSYVZw5v84wQ4BA6EMn8uGyNOGWUVHZutehxKryvY4xp48= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - misc teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8bec0da71becad44414c54979f64c9ef0e7099c6 timestamp: 2026-02-12_15:38:14 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3932482 2026-02-12T19:01:23.303 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa; will attempt to use it 2026-02-12T19:01:23.304 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks 2026-02-12T19:01:23.304 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-12T19:01:23.305 INFO:teuthology.task.internal:Checking packages... 2026-02-12T19:01:23.305 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'afefaa40503a4bf024e96f97dd7f852b75f00ad6' 2026-02-12T19:01:23.306 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-12T19:01:23.306 INFO:teuthology.packaging:ref: None 2026-02-12T19:01:23.306 INFO:teuthology.packaging:tag: None 2026-02-12T19:01:23.306 INFO:teuthology.packaging:branch: wip-yuri4-testing-2026-02-11-2126 2026-02-12T19:01:23.306 INFO:teuthology.packaging:sha1: afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-12T19:01:23.306 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri4-testing-2026-02-11-2126 2026-02-12T19:01:23.432 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5230-gafefaa40-1jammy 2026-02-12T19:01:23.433 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-12T19:01:23.444 INFO:teuthology.task.internal:no buildpackages task found 2026-02-12T19:01:23.444 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-12T19:01:23.458 INFO:teuthology.task.internal:Saving configuration 2026-02-12T19:01:23.466 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-12T19:01:23.479 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-12T19:01:23.557 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial010.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/46611', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-12 18:58:07.005885', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPYzChrNtE4u3DWqKiIoPiUD/IusLSRsdt6eqXDybm0S'} 2026-02-12T19:01:23.557 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-12T19:01:23.568 INFO:teuthology.task.internal:roles: ubuntu@trial010.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-12T19:01:23.568 INFO:teuthology.run_tasks:Running task console_log... 2026-02-12T19:01:23.648 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x76f99b7dfba0>, signals=[15]) 2026-02-12T19:01:23.648 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-12T19:01:23.660 INFO:teuthology.task.internal:Opening connections... 2026-02-12T19:01:23.660 DEBUG:teuthology.task.internal:connecting to ubuntu@trial010.front.sepia.ceph.com 2026-02-12T19:01:23.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial010.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T19:01:23.725 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-12T19:01:23.738 DEBUG:teuthology.orchestra.run.trial010:> uname -m 2026-02-12T19:01:23.803 INFO:teuthology.orchestra.run.trial010.stdout:x86_64 2026-02-12T19:01:23.803 DEBUG:teuthology.orchestra.run.trial010:> cat /etc/os-release 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:NAME="Ubuntu" 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:VERSION_ID="22.04" 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:VERSION_CODENAME=jammy 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:ID=ubuntu 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:ID_LIKE=debian 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-12T19:01:23.849 INFO:teuthology.orchestra.run.trial010.stdout:UBUNTU_CODENAME=jammy 2026-02-12T19:01:23.850 INFO:teuthology.lock.ops:Updating trial010.front.sepia.ceph.com on lock server 2026-02-12T19:01:23.936 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-12T19:01:23.949 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-12T19:01:23.961 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-12T19:01:23.961 DEBUG:teuthology.orchestra.run.trial010:> test '!' -e /home/ubuntu/cephtest 2026-02-12T19:01:23.965 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-12T19:01:23.978 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-12T19:01:23.978 DEBUG:teuthology.orchestra.run.trial010:> test -z $(ls -A /var/lib/ceph) 2026-02-12T19:01:24.013 INFO:teuthology.orchestra.run.trial010.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-12T19:01:24.014 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-12T19:01:24.115 INFO:teuthology.run_tasks:Running task kernel... 2026-02-12T19:01:24.135 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-12T19:01:24.135 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-12T19:01:24.135 DEBUG:teuthology.orchestra.run.trial010:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-12T19:01:24.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T19:01:24.138 DEBUG:teuthology.orchestra.run.trial010:> uname -r 2026-02-12T19:01:24.184 INFO:teuthology.orchestra.run.trial010.stdout:5.15.0-164-generic 2026-02-12T19:01:24.184 INFO:teuthology.task.kernel:Running kernel on trial010: 5.15.0-164-generic 2026-02-12T19:01:24.184 DEBUG:teuthology.orchestra.run.trial010:> sudo apt-get clean 2026-02-12T19:01:24.246 DEBUG:teuthology.orchestra.run.trial010:> sudo apt-get update 2026-02-12T19:01:24.428 INFO:teuthology.orchestra.run.trial010.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-12T19:01:24.436 INFO:teuthology.orchestra.run.trial010.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-12T19:01:24.473 INFO:teuthology.orchestra.run.trial010.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-12T19:01:24.489 INFO:teuthology.orchestra.run.trial010.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-12T19:01:24.571 INFO:teuthology.orchestra.run.trial010.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3220 kB] 2026-02-12T19:01:24.637 INFO:teuthology.orchestra.run.trial010.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-12T19:01:24.638 INFO:teuthology.orchestra.run.trial010.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1254 kB] 2026-02-12T19:01:24.645 INFO:teuthology.orchestra.run.trial010.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [313 kB] 2026-02-12T19:01:24.647 INFO:teuthology.orchestra.run.trial010.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.2 kB] 2026-02-12T19:01:24.649 INFO:teuthology.orchestra.run.trial010.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2959 kB] 2026-02-12T19:01:24.666 INFO:teuthology.orchestra.run.trial010.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [425 kB] 2026-02-12T19:01:24.668 INFO:teuthology.orchestra.run.trial010.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-12T19:01:24.669 INFO:teuthology.orchestra.run.trial010.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1016 kB] 2026-02-12T19:01:24.674 INFO:teuthology.orchestra.run.trial010.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/universe Translation-en [224 kB] 2026-02-12T19:01:24.676 INFO:teuthology.orchestra.run.trial010.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.6 kB] 2026-02-12T19:01:24.676 INFO:teuthology.orchestra.run.trial010.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5089 kB] 2026-02-12T19:01:24.692 INFO:teuthology.orchestra.run.trial010.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [963 kB] 2026-02-12T19:01:28.004 INFO:teuthology.orchestra.run.trial010.stdout:Fetched 15.9 MB in 1s (12.9 MB/s) 2026-02-12T19:01:28.539 INFO:teuthology.orchestra.run.trial010.stdout:Reading package lists... 2026-02-12T19:01:28.553 DEBUG:teuthology.orchestra.run.trial010:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-12T19:01:28.587 INFO:teuthology.orchestra.run.trial010.stdout:Reading package lists... 2026-02-12T19:01:28.695 INFO:teuthology.orchestra.run.trial010.stdout:Building dependency tree... 2026-02-12T19:01:28.695 INFO:teuthology.orchestra.run.trial010.stdout:Reading state information... 2026-02-12T19:01:28.784 INFO:teuthology.orchestra.run.trial010.stdout:The following packages were automatically installed and are no longer required: 2026-02-12T19:01:28.784 INFO:teuthology.orchestra.run.trial010.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-12T19:01:28.784 INFO:teuthology.orchestra.run.trial010.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-12T19:01:28.784 INFO:teuthology.orchestra.run.trial010.stdout: sg3-utils-udev 2026-02-12T19:01:28.784 INFO:teuthology.orchestra.run.trial010.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-12T19:01:28.784 INFO:teuthology.orchestra.run.trial010.stdout:The following additional packages will be installed: 2026-02-12T19:01:28.784 INFO:teuthology.orchestra.run.trial010.stdout: linux-generic linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-12T19:01:28.784 INFO:teuthology.orchestra.run.trial010.stdout: linux-headers-generic linux-image-5.15.0-170-generic 2026-02-12T19:01:28.784 INFO:teuthology.orchestra.run.trial010.stdout: linux-modules-5.15.0-170-generic linux-modules-extra-5.15.0-170-generic 2026-02-12T19:01:28.785 INFO:teuthology.orchestra.run.trial010.stdout:Suggested packages: 2026-02-12T19:01:28.785 INFO:teuthology.orchestra.run.trial010.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2026-02-12T19:01:28.797 INFO:teuthology.orchestra.run.trial010.stdout:The following NEW packages will be installed: 2026-02-12T19:01:28.797 INFO:teuthology.orchestra.run.trial010.stdout: linux-headers-5.15.0-170 linux-headers-5.15.0-170-generic 2026-02-12T19:01:28.797 INFO:teuthology.orchestra.run.trial010.stdout: linux-image-5.15.0-170-generic linux-modules-5.15.0-170-generic 2026-02-12T19:01:28.797 INFO:teuthology.orchestra.run.trial010.stdout: linux-modules-extra-5.15.0-170-generic 2026-02-12T19:01:28.797 INFO:teuthology.orchestra.run.trial010.stdout:The following packages will be upgraded: 2026-02-12T19:01:28.797 INFO:teuthology.orchestra.run.trial010.stdout: linux-generic linux-headers-generic linux-image-generic 2026-02-12T19:01:29.299 INFO:teuthology.orchestra.run.trial010.stdout:3 upgraded, 5 newly installed, 0 to remove and 37 not upgraded. 2026-02-12T19:01:29.299 INFO:teuthology.orchestra.run.trial010.stdout:Need to get 113 MB of archives. 2026-02-12T19:01:29.299 INFO:teuthology.orchestra.run.trial010.stdout:After this operation, 585 MB of additional disk space will be used. 2026-02-12T19:01:29.299 INFO:teuthology.orchestra.run.trial010.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-170-generic amd64 5.15.0-170.180 [22.7 MB] 2026-02-12T19:01:30.691 INFO:teuthology.orchestra.run.trial010.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-170-generic amd64 5.15.0-170.180 [11.6 MB] 2026-02-12T19:01:31.034 INFO:teuthology.orchestra.run.trial010.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-170-generic amd64 5.15.0-170.180 [64.0 MB] 2026-02-12T19:01:32.928 INFO:teuthology.orchestra.run.trial010.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.170.159 [1654 B] 2026-02-12T19:01:33.330 INFO:teuthology.orchestra.run.trial010.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.170.159 [2474 B] 2026-02-12T19:01:33.425 INFO:teuthology.orchestra.run.trial010.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170 all 5.15.0-170.180 [12.4 MB] 2026-02-12T19:01:34.332 INFO:teuthology.orchestra.run.trial010.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-170-generic amd64 5.15.0-170.180 [2828 kB] 2026-02-12T19:01:34.414 INFO:teuthology.orchestra.run.trial010.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.170.159 [2316 B] 2026-02-12T19:01:34.613 INFO:teuthology.orchestra.run.trial010.stdout:Fetched 113 MB in 6s (20.2 MB/s) 2026-02-12T19:01:34.632 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package linux-modules-5.15.0-170-generic. 2026-02-12T19:01:34.797 INFO:teuthology.orchestra.run.trial010.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 131954 files and directories currently installed.) 2026-02-12T19:01:34.798 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../0-linux-modules-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-12T19:01:34.800 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-12T19:01:35.151 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package linux-image-5.15.0-170-generic. 2026-02-12T19:01:35.158 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../1-linux-image-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-12T19:01:35.175 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-12T19:01:35.215 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-170-generic. 2026-02-12T19:01:35.222 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-12T19:01:35.222 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-12T19:01:36.324 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../3-linux-generic_5.15.0.170.159_amd64.deb ... 2026-02-12T19:01:36.324 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking linux-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-12T19:01:36.347 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.170.159_amd64.deb ... 2026-02-12T19:01:36.347 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking linux-image-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-12T19:01:36.366 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package linux-headers-5.15.0-170. 2026-02-12T19:01:36.373 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../5-linux-headers-5.15.0-170_5.15.0-170.180_all.deb ... 2026-02-12T19:01:36.373 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-12T19:01:37.697 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package linux-headers-5.15.0-170-generic. 2026-02-12T19:01:37.709 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../6-linux-headers-5.15.0-170-generic_5.15.0-170.180_amd64.deb ... 2026-02-12T19:01:37.709 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-12T19:01:38.305 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.170.159_amd64.deb ... 2026-02-12T19:01:38.306 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking linux-headers-generic (5.15.0.170.159) over (5.15.0.164.159) ... 2026-02-12T19:01:38.331 INFO:teuthology.orchestra.run.trial010.stdout:Setting up linux-headers-5.15.0-170 (5.15.0-170.180) ... 2026-02-12T19:01:38.332 INFO:teuthology.orchestra.run.trial010.stdout:Setting up linux-headers-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-12T19:01:38.334 INFO:teuthology.orchestra.run.trial010.stdout:Setting up linux-headers-generic (5.15.0.170.159) ... 2026-02-12T19:01:38.335 INFO:teuthology.orchestra.run.trial010.stdout:Setting up linux-modules-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-12T19:01:39.128 INFO:teuthology.orchestra.run.trial010.stdout:Setting up linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-12T19:01:39.973 INFO:teuthology.orchestra.run.trial010.stdout:I: /boot/vmlinuz is now a symlink to vmlinuz-5.15.0-170-generic 2026-02-12T19:01:39.974 INFO:teuthology.orchestra.run.trial010.stdout:I: /boot/initrd.img is now a symlink to initrd.img-5.15.0-170-generic 2026-02-12T19:01:39.979 INFO:teuthology.orchestra.run.trial010.stdout:Setting up linux-modules-extra-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-12T19:01:40.789 INFO:teuthology.orchestra.run.trial010.stdout:Setting up linux-image-generic (5.15.0.170.159) ... 2026-02-12T19:01:40.790 INFO:teuthology.orchestra.run.trial010.stdout:Setting up linux-generic (5.15.0.170.159) ... 2026-02-12T19:01:40.791 INFO:teuthology.orchestra.run.trial010.stdout:Processing triggers for linux-image-5.15.0-170-generic (5.15.0-170.180) ... 2026-02-12T19:01:40.798 INFO:teuthology.orchestra.run.trial010.stdout:/etc/kernel/postinst.d/initramfs-tools: 2026-02-12T19:01:40.798 INFO:teuthology.orchestra.run.trial010.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-170-generic 2026-02-12T19:01:47.884 INFO:teuthology.orchestra.run.trial010.stdout:W: Possible missing firmware /lib/firmware/ast_dp501_fw.bin for module ast 2026-02-12T19:01:54.500 INFO:teuthology.orchestra.run.trial010.stdout:/etc/kernel/postinst.d/zz-update-grub: 2026-02-12T19:01:54.501 INFO:teuthology.orchestra.run.trial010.stdout:Sourcing file `/etc/default/grub' 2026-02-12T19:01:54.520 INFO:teuthology.orchestra.run.trial010.stdout:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-12T19:01:54.521 INFO:teuthology.orchestra.run.trial010.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-12T19:01:54.523 INFO:teuthology.orchestra.run.trial010.stdout:Generating grub configuration file ... 2026-02-12T19:01:54.605 INFO:teuthology.orchestra.run.trial010.stdout:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-12T19:01:54.612 INFO:teuthology.orchestra.run.trial010.stdout:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-12T19:01:54.791 INFO:teuthology.orchestra.run.trial010.stdout:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-12T19:01:54.797 INFO:teuthology.orchestra.run.trial010.stdout:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-12T19:01:54.870 INFO:teuthology.orchestra.run.trial010.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-12T19:01:54.870 INFO:teuthology.orchestra.run.trial010.stdout:Systems on them will not be added to the GRUB boot configuration. 2026-02-12T19:01:54.870 INFO:teuthology.orchestra.run.trial010.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-12T19:01:54.872 INFO:teuthology.orchestra.run.trial010.stdout:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-12T19:01:54.882 INFO:teuthology.orchestra.run.trial010.stdout:done 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout:Pending kernel upgrade! 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout:Running kernel version: 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout: 5.15.0-164-generic 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout:Diagnostics: 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout: The currently running kernel version is not the expected kernel version 5.15.0-170-generic. 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout:Restarting the system to load the new kernel will not be handled automatically, so you should consider rebooting. [Return] 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout:The processor microcode seems to be up-to-date. 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout:No services need to be restarted. 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout:No containers need to be restarted. 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout:No user sessions are running outdated binaries. 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:01:55.285 INFO:teuthology.orchestra.run.trial010.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-12T19:01:55.940 DEBUG:teuthology.orchestra.run.trial010:> dpkg -s linux-image-generic 2026-02-12T19:01:55.992 INFO:teuthology.orchestra.run.trial010.stdout:Package: linux-image-generic 2026-02-12T19:01:55.992 INFO:teuthology.orchestra.run.trial010.stdout:Status: install ok installed 2026-02-12T19:01:55.992 INFO:teuthology.orchestra.run.trial010.stdout:Priority: optional 2026-02-12T19:01:55.992 INFO:teuthology.orchestra.run.trial010.stdout:Section: kernel 2026-02-12T19:01:55.992 INFO:teuthology.orchestra.run.trial010.stdout:Installed-Size: 21 2026-02-12T19:01:55.992 INFO:teuthology.orchestra.run.trial010.stdout:Maintainer: Ubuntu Kernel Team 2026-02-12T19:01:55.992 INFO:teuthology.orchestra.run.trial010.stdout:Architecture: amd64 2026-02-12T19:01:55.992 INFO:teuthology.orchestra.run.trial010.stdout:Source: linux-meta 2026-02-12T19:01:55.993 INFO:teuthology.orchestra.run.trial010.stdout:Version: 5.15.0.170.159 2026-02-12T19:01:55.993 INFO:teuthology.orchestra.run.trial010.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-12T19:01:55.993 INFO:teuthology.orchestra.run.trial010.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-12T19:01:55.993 INFO:teuthology.orchestra.run.trial010.stdout:Recommends: thermald 2026-02-12T19:01:55.993 INFO:teuthology.orchestra.run.trial010.stdout:Description: Generic Linux kernel image 2026-02-12T19:01:55.993 INFO:teuthology.orchestra.run.trial010.stdout: This package will always depend on the latest generic kernel image 2026-02-12T19:01:55.993 INFO:teuthology.orchestra.run.trial010.stdout: available. 2026-02-12T19:01:55.993 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-164-generic Expected: 5.15.0-170-generic 2026-02-12T19:01:55.994 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-12T19:01:55.994 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-12T19:01:55.994 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-12T19:01:55.994 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial010.front.sepia.ceph.com, path=None, version=distro) 2026-02-12T19:01:55.994 DEBUG:teuthology.orchestra.run.trial010:> sudo apt-get clean 2026-02-12T19:01:56.074 DEBUG:teuthology.orchestra.run.trial010:> sudo apt-get update 2026-02-12T19:01:56.564 INFO:teuthology.orchestra.run.trial010.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-12T19:01:56.646 INFO:teuthology.orchestra.run.trial010.stdout:Hit:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-12T19:01:56.735 INFO:teuthology.orchestra.run.trial010.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-12T19:01:56.823 INFO:teuthology.orchestra.run.trial010.stdout:Hit:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-12T19:01:57.488 INFO:teuthology.orchestra.run.trial010.stdout:Reading package lists... 2026-02-12T19:01:57.503 DEBUG:teuthology.orchestra.run.trial010:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-12T19:01:57.536 INFO:teuthology.orchestra.run.trial010.stdout:Reading package lists... 2026-02-12T19:01:57.647 INFO:teuthology.orchestra.run.trial010.stdout:Building dependency tree... 2026-02-12T19:01:57.648 INFO:teuthology.orchestra.run.trial010.stdout:Reading state information... 2026-02-12T19:01:57.735 INFO:teuthology.orchestra.run.trial010.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-12T19:01:57.736 INFO:teuthology.orchestra.run.trial010.stdout:linux-image-generic set to manually installed. 2026-02-12T19:01:57.736 INFO:teuthology.orchestra.run.trial010.stdout:The following packages were automatically installed and are no longer required: 2026-02-12T19:01:57.736 INFO:teuthology.orchestra.run.trial010.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-12T19:01:57.736 INFO:teuthology.orchestra.run.trial010.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-12T19:01:57.736 INFO:teuthology.orchestra.run.trial010.stdout: sg3-utils-udev 2026-02-12T19:01:57.736 INFO:teuthology.orchestra.run.trial010.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-12T19:01:57.756 INFO:teuthology.orchestra.run.trial010.stdout:0 upgraded, 0 newly installed, 0 to remove and 37 not upgraded. 2026-02-12T19:01:57.758 DEBUG:teuthology.orchestra.run.trial010:> dpkg -s linux-image-generic 2026-02-12T19:01:57.811 INFO:teuthology.orchestra.run.trial010.stdout:Package: linux-image-generic 2026-02-12T19:01:57.811 INFO:teuthology.orchestra.run.trial010.stdout:Status: install ok installed 2026-02-12T19:01:57.811 INFO:teuthology.orchestra.run.trial010.stdout:Priority: optional 2026-02-12T19:01:57.811 INFO:teuthology.orchestra.run.trial010.stdout:Section: kernel 2026-02-12T19:01:57.811 INFO:teuthology.orchestra.run.trial010.stdout:Installed-Size: 21 2026-02-12T19:01:57.811 INFO:teuthology.orchestra.run.trial010.stdout:Maintainer: Ubuntu Kernel Team 2026-02-12T19:01:57.811 INFO:teuthology.orchestra.run.trial010.stdout:Architecture: amd64 2026-02-12T19:01:57.811 INFO:teuthology.orchestra.run.trial010.stdout:Source: linux-meta 2026-02-12T19:01:57.811 INFO:teuthology.orchestra.run.trial010.stdout:Version: 5.15.0.170.159 2026-02-12T19:01:57.812 INFO:teuthology.orchestra.run.trial010.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-12T19:01:57.812 INFO:teuthology.orchestra.run.trial010.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-12T19:01:57.812 INFO:teuthology.orchestra.run.trial010.stdout:Recommends: thermald 2026-02-12T19:01:57.812 INFO:teuthology.orchestra.run.trial010.stdout:Description: Generic Linux kernel image 2026-02-12T19:01:57.812 INFO:teuthology.orchestra.run.trial010.stdout: This package will always depend on the latest generic kernel image 2026-02-12T19:01:57.812 INFO:teuthology.orchestra.run.trial010.stdout: available. 2026-02-12T19:01:57.812 DEBUG:teuthology.orchestra.run.trial010:> mktemp 2026-02-12T19:01:57.858 INFO:teuthology.orchestra.run.trial010.stdout:/tmp/tmp.EQibC3O2zT 2026-02-12T19:01:57.858 DEBUG:teuthology.orchestra.run.trial010:> sudo cp /boot/grub/grub.cfg /tmp/tmp.EQibC3O2zT 2026-02-12T19:01:57.910 DEBUG:teuthology.orchestra.run.trial010:> sudo chmod 0666 /tmp/tmp.EQibC3O2zT 2026-02-12T19:01:58.012 DEBUG:teuthology.orchestra.remote:trial010:/tmp/tmp.EQibC3O2zT is 7KB 2026-02-12T19:01:58.066 DEBUG:teuthology.orchestra.run.trial010:> rm -fr /tmp/tmp.EQibC3O2zT 2026-02-12T19:01:58.070 DEBUG:teuthology.orchestra.run.trial010:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2026-02-12T19:01:58.122 DEBUG:teuthology.orchestra.run.trial010:> set -ex 2026-02-12T19:01:58.122 DEBUG:teuthology.orchestra.run.trial010:> sudo dd of=/etc/grub.d/01_ceph_kernel 2026-02-12T19:01:58.122 DEBUG:teuthology.orchestra.run.trial010:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2026-02-12T19:01:58.179 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-170-generic 2026-02-12T19:01:58.179 DEBUG:teuthology.orchestra.run.trial010:> sudo update-grub 2026-02-12T19:01:58.411 INFO:teuthology.orchestra.run.trial010.stderr:Sourcing file `/etc/default/grub' 2026-02-12T19:01:58.430 INFO:teuthology.orchestra.run.trial010.stderr:Sourcing file `/etc/default/grub.d/50-curtin-settings.cfg' 2026-02-12T19:01:58.431 INFO:teuthology.orchestra.run.trial010.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2026-02-12T19:01:58.433 INFO:teuthology.orchestra.run.trial010.stderr:Generating grub configuration file ... 2026-02-12T19:01:58.516 INFO:teuthology.orchestra.run.trial010.stderr:Found linux image: /boot/vmlinuz-5.15.0-170-generic 2026-02-12T19:01:58.522 INFO:teuthology.orchestra.run.trial010.stderr:Found initrd image: /boot/initrd.img-5.15.0-170-generic 2026-02-12T19:01:58.696 INFO:teuthology.orchestra.run.trial010.stderr:Found linux image: /boot/vmlinuz-5.15.0-164-generic 2026-02-12T19:01:58.701 INFO:teuthology.orchestra.run.trial010.stderr:Found initrd image: /boot/initrd.img-5.15.0-164-generic 2026-02-12T19:01:58.769 INFO:teuthology.orchestra.run.trial010.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2026-02-12T19:01:58.769 INFO:teuthology.orchestra.run.trial010.stderr:Systems on them will not be added to the GRUB boot configuration. 2026-02-12T19:01:58.769 INFO:teuthology.orchestra.run.trial010.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2026-02-12T19:01:58.771 INFO:teuthology.orchestra.run.trial010.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-12T19:01:58.778 INFO:teuthology.orchestra.run.trial010.stderr:done 2026-02-12T19:01:58.780 DEBUG:teuthology.orchestra.run.trial010:> sudo shutdown -r now 2026-02-12T19:02:28.809 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-12T19:02:28.809 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial010.front.sepia.ceph.com' 2026-02-12T19:02:28.810 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial010.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T19:03:28.855 DEBUG:teuthology.orchestra.remote:timed out 2026-02-12T19:03:37.865 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial010.front.sepia.ceph.com' 2026-02-12T19:03:37.866 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial010.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T19:03:37.992 DEBUG:teuthology.orchestra.run.trial010:> true 2026-02-12T19:03:38.387 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial010.front.sepia.ceph.com' 2026-02-12T19:03:38.388 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-170-generic"... 2026-02-12T19:03:38.388 DEBUG:teuthology.orchestra.run.trial010:> uname -r 2026-02-12T19:03:38.433 INFO:teuthology.orchestra.run.trial010.stdout:5.15.0-170-generic 2026-02-12T19:03:38.434 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-170-generic vs 5.15.0-170-generic 2026-02-12T19:03:38.434 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-12T19:03:38.434 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2026-02-12T19:03:39.434 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-12T19:03:39.435 DEBUG:teuthology.orchestra.run.trial010:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-12T19:03:39.448 INFO:teuthology.orchestra.run.trial010.stdout:ttyS1 2026-02-12T19:03:39.468 DEBUG:teuthology.parallel:result is None 2026-02-12T19:03:39.468 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-12T19:03:39.476 INFO:teuthology.task.internal:Creating test directory... 2026-02-12T19:03:39.476 DEBUG:teuthology.orchestra.run.trial010:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-12T19:03:39.514 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-12T19:03:39.525 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-12T19:03:39.538 INFO:teuthology.task.internal:Creating archive directory... 2026-02-12T19:03:39.539 DEBUG:teuthology.orchestra.run.trial010:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-12T19:03:39.564 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-12T19:03:39.577 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-12T19:03:39.577 DEBUG:teuthology.orchestra.run.trial010:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-12T19:03:39.611 INFO:teuthology.orchestra.run.trial010.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:03:39.617 INFO:teuthology.orchestra.run.trial010.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:03:39.619 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-12T19:03:39.630 INFO:teuthology.task.internal:Configuring sudo... 2026-02-12T19:03:39.630 DEBUG:teuthology.orchestra.run.trial010:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-12T19:03:39.670 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-12T19:03:39.687 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-12T19:03:39.687 DEBUG:teuthology.orchestra.run.trial010:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-12T19:03:39.718 DEBUG:teuthology.orchestra.run.trial010:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T19:03:39.762 DEBUG:teuthology.orchestra.run.trial010:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T19:03:39.806 DEBUG:teuthology.orchestra.run.trial010:> set -ex 2026-02-12T19:03:39.806 DEBUG:teuthology.orchestra.run.trial010:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-12T19:03:39.854 DEBUG:teuthology.orchestra.run.trial010:> sudo service rsyslog restart 2026-02-12T19:03:39.911 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-12T19:03:39.923 INFO:teuthology.task.internal:Starting timer... 2026-02-12T19:03:39.924 INFO:teuthology.run_tasks:Running task pcp... 2026-02-12T19:03:39.944 INFO:teuthology.run_tasks:Running task selinux... 2026-02-12T19:03:39.962 DEBUG:teuthology.task.selinux:Excluding trial010: OS 'ubuntu' does not support SELinux 2026-02-12T19:03:39.962 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-12T19:03:39.962 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-12T19:03:39.962 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-12T19:03:39.962 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-12T19:03:39.977 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-12T19:03:39.990 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-12T19:03:39.991 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial010.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-12T19:04:36.804 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial010.front.sepia.ceph.com')] 2026-02-12T19:04:36.806 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial010.front.sepia.ceph.com' 2026-02-12T19:04:36.808 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial010.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T19:04:36.914 DEBUG:teuthology.orchestra.run.trial010:> true 2026-02-12T19:04:36.994 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial010.front.sepia.ceph.com' 2026-02-12T19:04:36.994 INFO:teuthology.run_tasks:Running task clock... 2026-02-12T19:04:37.007 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-12T19:04:37.007 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-12T19:04:37.008 DEBUG:teuthology.orchestra.run.trial010:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T19:04:37.052 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-12T19:04:37.052 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: Command line: ntpd -gq 2026-02-12T19:04:37.052 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: ---------------------------------------------------- 2026-02-12T19:04:37.052 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: ntp-4 is maintained by Network Time Foundation, 2026-02-12T19:04:37.052 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-12T19:04:37.052 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: corporation. Support and training for ntp-4 are 2026-02-12T19:04:37.052 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: available at https://www.nwtime.org/support 2026-02-12T19:04:37.052 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: ---------------------------------------------------- 2026-02-12T19:04:37.053 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: proto: precision = 0.030 usec (-25) 2026-02-12T19:04:37.053 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: basedate set to 2022-02-04 2026-02-12T19:04:37.053 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: gps base set to 2022-02-06 (week 2196) 2026-02-12T19:04:37.053 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-12T19:04:37.054 INFO:teuthology.orchestra.run.trial010.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-12T19:04:37.054 INFO:teuthology.orchestra.run.trial010.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-12T19:04:37.055 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: restrict ::: KOD does nothing without LIMITED. 2026-02-12T19:04:37.055 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: Listen and drop on 0 v6wildcard [::]:123 2026-02-12T19:04:37.055 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-12T19:04:37.055 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: Listen normally on 2 lo 127.0.0.1:123 2026-02-12T19:04:37.055 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: Listen normally on 3 enp1s0f0 10.20.193.10:123 2026-02-12T19:04:37.055 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: Listen normally on 4 lo [::1]:123 2026-02-12T19:04:37.055 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:5e60%2]:123 2026-02-12T19:04:37.055 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:37 ntpd[4893]: Listening on routing socket on fd #22 for interface updates 2026-02-12T19:04:38.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:38.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:38.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:38.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:38.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:38.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:38.069 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:38.069 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:38.070 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:38.070 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:38.094 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:38.095 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:38.095 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:38.095 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:38.096 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:38.096 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:38.096 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:38.096 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:38 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:39.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:39 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:39.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:39 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:39.122 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:39 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:39.122 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:39 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:39.122 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:39 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:39.122 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:39 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:40.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:40.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:40.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:40.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:40.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:40.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:40.070 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:40.070 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:40.070 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:40.070 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:40.089 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:40.089 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:40.090 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:40.090 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:40.094 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:40.094 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:40.094 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:40.095 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:40 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:41.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:41 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:41.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:41 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:41.121 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:41 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:41.122 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:41 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:41.122 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:41 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:41.122 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:41 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:42.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:42.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:42.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:42.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:42.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:42.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:42.072 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:42.072 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:42.072 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:42.073 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:42.089 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:42.089 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:42.090 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:42.090 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:42.094 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:42.094 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:42.094 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:42.094 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:42 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:43.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:43 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:43.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:43 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:43.122 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:43 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:43.123 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:43 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:43.123 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:43 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:43.123 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:43 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:44.052 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:44.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:44.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:44.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:44.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:44.053 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:44.068 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-12T19:04:44.068 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: can't open /var/log/ntpstats/rawstats.20260212: Permission denied 2026-02-12T19:04:44.068 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-12T19:04:44.068 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: can't open /var/log/ntpstats/peerstats.20260212: Permission denied 2026-02-12T19:04:44.069 INFO:teuthology.orchestra.run.trial010.stdout:12 Feb 19:04:44 ntpd[4893]: ntpd: time set -0.678645 s 2026-02-12T19:04:44.069 INFO:teuthology.orchestra.run.trial010.stdout:ntpd: time set -0.678645s 2026-02-12T19:04:44.069 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-12T19:04:44.069 INFO:teuthology.orchestra.run.trial010.stderr:12 Feb 19:04:44 ntpd[4893]: can't open /var/log/ntpstats/loopstats.20260212: Permission denied 2026-02-12T19:04:44.092 INFO:teuthology.orchestra.run.trial010.stdout:No association ID's returned 2026-02-12T19:04:44.093 INFO:teuthology.run_tasks:Running task install... 2026-02-12T19:04:44.106 DEBUG:teuthology.task.install:project ceph 2026-02-12T19:04:44.106 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'}} 2026-02-12T19:04:44.107 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'} 2026-02-12T19:04:44.107 INFO:teuthology.task.install:Using flavor: default 2026-02-12T19:04:44.116 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-12T19:04:44.116 INFO:teuthology.task.install:extra packages: [] 2026-02-12T19:04:44.117 DEBUG:teuthology.orchestra.run.trial010:> sudo apt-key list | grep Ceph 2026-02-12T19:04:44.188 INFO:teuthology.orchestra.run.trial010.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-12T19:04:44.216 INFO:teuthology.orchestra.run.trial010.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-12T19:04:44.217 INFO:teuthology.orchestra.run.trial010.stdout:uid [ unknown] Ceph.com (release key) 2026-02-12T19:04:44.218 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-osd, ceph-osd-classic, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-12T19:04:44.219 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-12T19:04:44.382 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default/ 2026-02-12T19:04:44.382 INFO:teuthology.task.install.deb:Package version is 20.3.0-5230-gafefaa40-1jammy 2026-02-12T19:04:44.772 DEBUG:teuthology.orchestra.run.trial010:> set -ex 2026-02-12T19:04:44.772 DEBUG:teuthology.orchestra.run.trial010:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-12T19:04:44.783 DEBUG:teuthology.orchestra.run.trial010:> sudo apt-get update 2026-02-12T19:04:45.187 INFO:teuthology.orchestra.run.trial010.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy InRelease 2026-02-12T19:04:45.255 INFO:teuthology.orchestra.run.trial010.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy Release [7689 B] 2026-02-12T19:04:45.292 INFO:teuthology.orchestra.run.trial010.stdout:Hit:3 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-12T19:04:45.323 INFO:teuthology.orchestra.run.trial010.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-12T19:04:45.376 INFO:teuthology.orchestra.run.trial010.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-12T19:04:45.391 INFO:teuthology.orchestra.run.trial010.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 Packages [19.2 kB] 2026-02-12T19:04:45.464 INFO:teuthology.orchestra.run.trial010.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-12T19:04:45.553 INFO:teuthology.orchestra.run.trial010.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-12T19:04:45.697 INFO:teuthology.orchestra.run.trial010.stdout:Fetched 26.9 kB in 1s (35.6 kB/s) 2026-02-12T19:04:46.229 INFO:teuthology.orchestra.run.trial010.stdout:Reading package lists... 2026-02-12T19:04:46.243 DEBUG:teuthology.orchestra.run.trial010:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.3.0-5230-gafefaa40-1jammy cephadm=20.3.0-5230-gafefaa40-1jammy ceph-mds=20.3.0-5230-gafefaa40-1jammy ceph-mgr=20.3.0-5230-gafefaa40-1jammy ceph-osd=20.3.0-5230-gafefaa40-1jammy ceph-osd-classic=20.3.0-5230-gafefaa40-1jammy ceph-common=20.3.0-5230-gafefaa40-1jammy ceph-fuse=20.3.0-5230-gafefaa40-1jammy ceph-test=20.3.0-5230-gafefaa40-1jammy ceph-volume=20.3.0-5230-gafefaa40-1jammy radosgw=20.3.0-5230-gafefaa40-1jammy python3-rados=20.3.0-5230-gafefaa40-1jammy python3-rgw=20.3.0-5230-gafefaa40-1jammy python3-cephfs=20.3.0-5230-gafefaa40-1jammy python3-rbd=20.3.0-5230-gafefaa40-1jammy libcephfs2=20.3.0-5230-gafefaa40-1jammy libcephfs-dev=20.3.0-5230-gafefaa40-1jammy librados2=20.3.0-5230-gafefaa40-1jammy librbd1=20.3.0-5230-gafefaa40-1jammy rbd-fuse=20.3.0-5230-gafefaa40-1jammy 2026-02-12T19:04:46.277 INFO:teuthology.orchestra.run.trial010.stdout:Reading package lists... 2026-02-12T19:04:46.385 INFO:teuthology.orchestra.run.trial010.stdout:Building dependency tree... 2026-02-12T19:04:46.385 INFO:teuthology.orchestra.run.trial010.stdout:Reading state information... 2026-02-12T19:04:46.474 INFO:teuthology.orchestra.run.trial010.stdout:The following packages were automatically installed and are no longer required: 2026-02-12T19:04:46.474 INFO:teuthology.orchestra.run.trial010.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-12T19:04:46.475 INFO:teuthology.orchestra.run.trial010.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-12T19:04:46.475 INFO:teuthology.orchestra.run.trial010.stdout: sg3-utils-udev 2026-02-12T19:04:46.475 INFO:teuthology.orchestra.run.trial010.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-12T19:04:46.475 INFO:teuthology.orchestra.run.trial010.stdout:The following additional packages will be installed: 2026-02-12T19:04:46.475 INFO:teuthology.orchestra.run.trial010.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-12T19:04:46.475 INFO:teuthology.orchestra.run.trial010.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon jq libcephfs-daemon 2026-02-12T19:04:46.475 INFO:teuthology.orchestra.run.trial010.stdout: libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-12T19:04:46.475 INFO:teuthology.orchestra.run.trial010.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-12T19:04:46.475 INFO:teuthology.orchestra.run.trial010.stdout: libradosstriper1 librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2026-02-12T19:04:46.475 INFO:teuthology.orchestra.run.trial010.stdout: nvme-cli python-asyncssh-doc python3-asyncssh python3-cachetools 2026-02-12T19:04:46.475 INFO:teuthology.orchestra.run.trial010.stdout: python3-ceph-argparse python3-ceph-common python3-cheroot python3-cherrypy3 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout: python3-google-auth python3-iniconfig python3-jaraco.classes 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout: python3-joblib python3-kubernetes python3-natsort python3-pluggy 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout: python3-portend python3-prettytable python3-psutil python3-py 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout: python3-pygments python3-pytest python3-repoze.lru python3-requests-oauthlib 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout: python3-routes python3-rsa python3-simplejson python3-sklearn 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout: python3-sklearn-lib python3-tempora python3-threadpoolctl python3-toml 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout: python3-wcwidth python3-webob python3-websocket python3-zc.lockfile 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout: qttranslations5-l10n socat xmlstarlet 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout:Suggested packages: 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-12T19:04:46.476 INFO:teuthology.orchestra.run.trial010.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-12T19:04:46.477 INFO:teuthology.orchestra.run.trial010.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-12T19:04:46.477 INFO:teuthology.orchestra.run.trial010.stdout:Recommended packages: 2026-02-12T19:04:46.477 INFO:teuthology.orchestra.run.trial010.stdout: btrfs-tools 2026-02-12T19:04:46.510 INFO:teuthology.orchestra.run.trial010.stdout:The following NEW packages will be installed: 2026-02-12T19:04:46.510 INFO:teuthology.orchestra.run.trial010.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-12T19:04:46.510 INFO:teuthology.orchestra.run.trial010.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-12T19:04:46.510 INFO:teuthology.orchestra.run.trial010.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-osd-classic ceph-test 2026-02-12T19:04:46.510 INFO:teuthology.orchestra.run.trial010.stdout: ceph-volume cephadm jq libcephfs-daemon libcephfs-dev libcephfs-proxy2 2026-02-12T19:04:46.510 INFO:teuthology.orchestra.run.trial010.stdout: libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 2026-02-12T19:04:46.510 INFO:teuthology.orchestra.run.trial010.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-12T19:04:46.510 INFO:teuthology.orchestra.run.trial010.stdout: librados2 libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-12T19:04:46.511 INFO:teuthology.orchestra.run.trial010.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-12T19:04:46.512 INFO:teuthology.orchestra.run.trial010.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-12T19:04:46.512 INFO:teuthology.orchestra.run.trial010.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-12T19:04:46.512 INFO:teuthology.orchestra.run.trial010.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-12T19:04:46.512 INFO:teuthology.orchestra.run.trial010.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-12T19:04:46.512 INFO:teuthology.orchestra.run.trial010.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-12T19:04:46.512 INFO:teuthology.orchestra.run.trial010.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-12T19:04:46.512 INFO:teuthology.orchestra.run.trial010.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-12T19:04:46.512 INFO:teuthology.orchestra.run.trial010.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-12T19:04:46.512 INFO:teuthology.orchestra.run.trial010.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-12T19:04:46.512 INFO:teuthology.orchestra.run.trial010.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-12T19:04:46.512 INFO:teuthology.orchestra.run.trial010.stdout: socat xmlstarlet 2026-02-12T19:04:46.610 INFO:teuthology.orchestra.run.trial010.stdout:0 upgraded, 87 newly installed, 0 to remove and 37 not upgraded. 2026-02-12T19:04:46.610 INFO:teuthology.orchestra.run.trial010.stdout:Need to get 701 MB of archives. 2026-02-12T19:04:46.610 INFO:teuthology.orchestra.run.trial010.stdout:After this operation, 2670 MB of additional disk space will be used. 2026-02-12T19:04:46.610 INFO:teuthology.orchestra.run.trial010.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-12T19:04:46.655 INFO:teuthology.orchestra.run.trial010.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-12T19:04:46.656 INFO:teuthology.orchestra.run.trial010.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-12T19:04:46.670 INFO:teuthology.orchestra.run.trial010.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-12T19:04:46.706 INFO:teuthology.orchestra.run.trial010.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-12T19:04:46.708 INFO:teuthology.orchestra.run.trial010.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-12T19:04:46.713 INFO:teuthology.orchestra.run.trial010.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-12T19:04:46.715 INFO:teuthology.orchestra.run.trial010.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-12T19:04:46.716 INFO:teuthology.orchestra.run.trial010.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-12T19:04:46.716 INFO:teuthology.orchestra.run.trial010.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-12T19:04:46.717 INFO:teuthology.orchestra.run.trial010.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-12T19:04:46.721 INFO:teuthology.orchestra.run.trial010.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-12T19:04:46.722 INFO:teuthology.orchestra.run.trial010.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-12T19:04:46.722 INFO:teuthology.orchestra.run.trial010.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-12T19:04:46.727 INFO:teuthology.orchestra.run.trial010.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-12T19:04:46.727 INFO:teuthology.orchestra.run.trial010.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-12T19:04:46.728 INFO:teuthology.orchestra.run.trial010.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-12T19:04:46.728 INFO:teuthology.orchestra.run.trial010.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-12T19:04:46.728 INFO:teuthology.orchestra.run.trial010.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-12T19:04:46.737 INFO:teuthology.orchestra.run.trial010.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-12T19:04:46.737 INFO:teuthology.orchestra.run.trial010.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-12T19:04:46.738 INFO:teuthology.orchestra.run.trial010.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-12T19:04:46.739 INFO:teuthology.orchestra.run.trial010.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-12T19:04:46.740 INFO:teuthology.orchestra.run.trial010.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-12T19:04:46.746 INFO:teuthology.orchestra.run.trial010.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-12T19:04:46.746 INFO:teuthology.orchestra.run.trial010.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-12T19:04:46.747 INFO:teuthology.orchestra.run.trial010.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-12T19:04:46.761 INFO:teuthology.orchestra.run.trial010.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-12T19:04:46.763 INFO:teuthology.orchestra.run.trial010.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-12T19:04:46.763 INFO:teuthology.orchestra.run.trial010.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-12T19:04:46.774 INFO:teuthology.orchestra.run.trial010.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-12T19:04:46.774 INFO:teuthology.orchestra.run.trial010.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-12T19:04:46.775 INFO:teuthology.orchestra.run.trial010.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-12T19:04:46.775 INFO:teuthology.orchestra.run.trial010.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-12T19:04:46.775 INFO:teuthology.orchestra.run.trial010.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-12T19:04:46.775 INFO:teuthology.orchestra.run.trial010.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-12T19:04:46.777 INFO:teuthology.orchestra.run.trial010.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-12T19:04:46.777 INFO:teuthology.orchestra.run.trial010.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-12T19:04:46.778 INFO:teuthology.orchestra.run.trial010.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-12T19:04:46.784 INFO:teuthology.orchestra.run.trial010.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-12T19:04:46.787 INFO:teuthology.orchestra.run.trial010.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-12T19:04:46.789 INFO:teuthology.orchestra.run.trial010.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-12T19:04:46.791 INFO:teuthology.orchestra.run.trial010.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-12T19:04:46.793 INFO:teuthology.orchestra.run.trial010.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-12T19:04:46.793 INFO:teuthology.orchestra.run.trial010.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-12T19:04:46.793 INFO:teuthology.orchestra.run.trial010.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-12T19:04:46.794 INFO:teuthology.orchestra.run.trial010.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-12T19:04:46.794 INFO:teuthology.orchestra.run.trial010.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-12T19:04:46.797 INFO:teuthology.orchestra.run.trial010.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-12T19:04:46.803 INFO:teuthology.orchestra.run.trial010.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-12T19:04:46.804 INFO:teuthology.orchestra.run.trial010.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-12T19:04:46.804 INFO:teuthology.orchestra.run.trial010.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-12T19:04:46.805 INFO:teuthology.orchestra.run.trial010.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-12T19:04:46.861 INFO:teuthology.orchestra.run.trial010.stdout:Get:54 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.3.0-5230-gafefaa40-1jammy [3798 kB] 2026-02-12T19:04:47.327 INFO:teuthology.orchestra.run.trial010.stdout:Get:55 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.3.0-5230-gafefaa40-1jammy [2910 kB] 2026-02-12T19:04:47.521 INFO:teuthology.orchestra.run.trial010.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.3.0-5230-gafefaa40-1jammy [882 kB] 2026-02-12T19:04:47.585 INFO:teuthology.orchestra.run.trial010.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.3.0-5230-gafefaa40-1jammy [359 kB] 2026-02-12T19:04:47.640 INFO:teuthology.orchestra.run.trial010.stdout:Get:58 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.3.0-5230-gafefaa40-1jammy [32.8 kB] 2026-02-12T19:04:47.640 INFO:teuthology.orchestra.run.trial010.stdout:Get:59 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.3.0-5230-gafefaa40-1jammy [225 kB] 2026-02-12T19:04:47.644 INFO:teuthology.orchestra.run.trial010.stdout:Get:60 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.3.0-5230-gafefaa40-1jammy [88.4 kB] 2026-02-12T19:04:47.646 INFO:teuthology.orchestra.run.trial010.stdout:Get:61 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.3.0-5230-gafefaa40-1jammy [339 kB] 2026-02-12T19:04:47.701 INFO:teuthology.orchestra.run.trial010.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.3.0-5230-gafefaa40-1jammy [9083 kB] 2026-02-12T19:04:48.371 INFO:teuthology.orchestra.run.trial010.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.3.0-5230-gafefaa40-1jammy [112 kB] 2026-02-12T19:04:48.423 INFO:teuthology.orchestra.run.trial010.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.3.0-5230-gafefaa40-1jammy [261 kB] 2026-02-12T19:04:48.428 INFO:teuthology.orchestra.run.trial010.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.3.0-5230-gafefaa40-1jammy [31.5 MB] 2026-02-12T19:04:50.722 INFO:teuthology.orchestra.run.trial010.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.3.0-5230-gafefaa40-1jammy [4982 kB] 2026-02-12T19:04:50.986 INFO:teuthology.orchestra.run.trial010.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.3.0-5230-gafefaa40-1jammy [293 kB] 2026-02-12T19:04:51.027 INFO:teuthology.orchestra.run.trial010.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.3.0-5230-gafefaa40-1jammy [127 kB] 2026-02-12T19:04:51.029 INFO:teuthology.orchestra.run.trial010.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.3.0-5230-gafefaa40-1jammy [922 kB] 2026-02-12T19:04:51.082 INFO:teuthology.orchestra.run.trial010.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.3.0-5230-gafefaa40-1jammy [7542 kB] 2026-02-12T19:04:51.433 INFO:teuthology.orchestra.run.trial010.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5230-gafefaa40-1jammy [468 MB] 2026-02-12T19:05:26.936 INFO:teuthology.orchestra.run.trial010.stdout:Ign:71 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5230-gafefaa40-1jammy 2026-02-12T19:05:27.004 INFO:teuthology.orchestra.run.trial010.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.3.0-5230-gafefaa40-1jammy [180 kB] 2026-02-12T19:05:27.027 INFO:teuthology.orchestra.run.trial010.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.3.0-5230-gafefaa40-1jammy [14.0 kB] 2026-02-12T19:05:27.027 INFO:teuthology.orchestra.run.trial010.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.3.0-5230-gafefaa40-1jammy [1030 kB] 2026-02-12T19:05:27.061 INFO:teuthology.orchestra.run.trial010.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.3.0-5230-gafefaa40-1jammy [2438 kB] 2026-02-12T19:05:27.095 INFO:teuthology.orchestra.run.trial010.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.3.0-5230-gafefaa40-1jammy [1068 kB] 2026-02-12T19:05:27.112 INFO:teuthology.orchestra.run.trial010.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.3.0-5230-gafefaa40-1jammy [190 kB] 2026-02-12T19:05:27.115 INFO:teuthology.orchestra.run.trial010.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.3.0-5230-gafefaa40-1jammy [54.5 MB] 2026-02-12T19:05:28.034 INFO:teuthology.orchestra.run.trial010.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.3.0-5230-gafefaa40-1jammy [8626 kB] 2026-02-12T19:05:28.150 INFO:teuthology.orchestra.run.trial010.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.3.0-5230-gafefaa40-1jammy [14.2 kB] 2026-02-12T19:05:28.150 INFO:teuthology.orchestra.run.trial010.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.3.0-5230-gafefaa40-1jammy [68.6 MB] 2026-02-12T19:05:29.308 INFO:teuthology.orchestra.run.trial010.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.3.0-5230-gafefaa40-1jammy [136 kB] 2026-02-12T19:05:29.309 INFO:teuthology.orchestra.run.trial010.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.3.0-5230-gafefaa40-1jammy [45.1 kB] 2026-02-12T19:05:29.310 INFO:teuthology.orchestra.run.trial010.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.3.0-5230-gafefaa40-1jammy [31.7 kB] 2026-02-12T19:05:29.310 INFO:teuthology.orchestra.run.trial010.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.3.0-5230-gafefaa40-1jammy [44.2 kB] 2026-02-12T19:05:29.312 INFO:teuthology.orchestra.run.trial010.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.3.0-5230-gafefaa40-1jammy [18.3 MB] 2026-02-12T19:05:29.579 INFO:teuthology.orchestra.run.trial010.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.3.0-5230-gafefaa40-1jammy [93.5 kB] 2026-02-12T19:05:29.792 INFO:teuthology.orchestra.run.trial010.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-yuri4-testing-2026-02-11-2126/afefaa40503a4bf024e96f97dd7f852b75f00ad6/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd-classic amd64 20.3.0-5230-gafefaa40-1jammy [468 MB] 2026-02-12T19:05:35.745 INFO:teuthology.orchestra.run.trial010.stdout:Fetched 584 MB in 49s (11.9 MB/s) 2026-02-12T19:05:35.761 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-12T19:05:35.920 INFO:teuthology.orchestra.run.trial010.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167438 files and directories currently installed.) 2026-02-12T19:05:35.922 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-12T19:05:35.923 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-12T19:05:35.944 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-12T19:05:35.955 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-12T19:05:35.955 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-12T19:05:35.971 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-12T19:05:35.983 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-12T19:05:35.984 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-12T19:05:36.003 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-12T19:05:36.011 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-12T19:05:36.017 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-12T19:05:36.050 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-12T19:05:36.060 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-12T19:05:36.060 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-12T19:05:36.077 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-12T19:05:36.086 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-12T19:05:36.087 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-12T19:05:36.106 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-12T19:05:36.114 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-12T19:05:36.115 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-12T19:05:36.131 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package librados2. 2026-02-12T19:05:36.139 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../07-librados2_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:36.139 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking librados2 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.182 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libnbd0. 2026-02-12T19:05:36.190 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-12T19:05:36.191 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-12T19:05:36.205 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package librbd1. 2026-02-12T19:05:36.213 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../09-librbd1_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:36.213 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking librbd1 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.250 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libcephfs2. 2026-02-12T19:05:36.258 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../10-libcephfs2_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:36.258 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libcephfs2 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.278 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-rados. 2026-02-12T19:05:36.286 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../11-python3-rados_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:36.287 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-rados (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.303 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-12T19:05:36.311 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.3.0-5230-gafefaa40-1jammy_all.deb ... 2026-02-12T19:05:36.312 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-ceph-argparse (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.325 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-cephfs. 2026-02-12T19:05:36.333 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../13-python3-cephfs_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:36.334 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-cephfs (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.350 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-12T19:05:36.358 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../14-python3-ceph-common_20.3.0-5230-gafefaa40-1jammy_all.deb ... 2026-02-12T19:05:36.359 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-ceph-common (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.377 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-12T19:05:36.385 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-12T19:05:36.385 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-12T19:05:36.401 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-prettytable. 2026-02-12T19:05:36.409 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-12T19:05:36.410 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-12T19:05:36.425 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-rbd. 2026-02-12T19:05:36.432 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../17-python3-rbd_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:36.433 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-rbd (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.450 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-12T19:05:36.458 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-12T19:05:36.458 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-12T19:05:36.476 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package librgw2. 2026-02-12T19:05:36.484 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../19-librgw2_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:36.484 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking librgw2 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.582 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-rgw. 2026-02-12T19:05:36.590 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../20-python3-rgw_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:36.590 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-rgw (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.606 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-12T19:05:36.614 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-12T19:05:36.614 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-12T19:05:36.629 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libradosstriper1. 2026-02-12T19:05:36.637 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../22-libradosstriper1_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:36.637 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libradosstriper1 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.654 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-common. 2026-02-12T19:05:36.662 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../23-ceph-common_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:36.662 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-common (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.919 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-base. 2026-02-12T19:05:36.926 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../24-ceph-base_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:36.931 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-base (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:36.990 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-12T19:05:36.998 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-12T19:05:36.998 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-12T19:05:37.013 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-cheroot. 2026-02-12T19:05:37.021 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-12T19:05:37.021 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-12T19:05:37.038 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-12T19:05:37.047 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-12T19:05:37.047 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-12T19:05:37.061 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-12T19:05:37.069 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-12T19:05:37.070 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-12T19:05:37.084 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-12T19:05:37.092 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-12T19:05:37.093 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-12T19:05:37.106 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-tempora. 2026-02-12T19:05:37.114 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-12T19:05:37.115 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-12T19:05:37.130 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-portend. 2026-02-12T19:05:37.138 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-12T19:05:37.138 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-12T19:05:37.153 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-12T19:05:37.161 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-12T19:05:37.161 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-12T19:05:37.175 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-12T19:05:37.183 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-12T19:05:37.184 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-12T19:05:37.207 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-natsort. 2026-02-12T19:05:37.215 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-12T19:05:37.215 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-12T19:05:37.230 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-12T19:05:37.238 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.3.0-5230-gafefaa40-1jammy_all.deb ... 2026-02-12T19:05:37.239 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-mgr-modules-core (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:37.270 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-12T19:05:37.278 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:37.278 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libsqlite3-mod-ceph (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:37.293 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-mgr. 2026-02-12T19:05:37.301 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../37-ceph-mgr_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:37.301 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-mgr (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:37.322 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-mon. 2026-02-12T19:05:37.330 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../38-ceph-mon_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:37.331 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-mon (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:37.394 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-12T19:05:37.402 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-12T19:05:37.402 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-12T19:05:37.418 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-osd-classic. 2026-02-12T19:05:37.426 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../40-ceph-osd-classic_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:37.426 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-osd-classic (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:41.291 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-osd. 2026-02-12T19:05:41.299 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../41-ceph-osd_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:41.299 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-osd (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:41.317 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph. 2026-02-12T19:05:41.325 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../42-ceph_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:41.326 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:41.340 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-fuse. 2026-02-12T19:05:41.352 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../43-ceph-fuse_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:41.352 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-fuse (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:41.374 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-mds. 2026-02-12T19:05:41.383 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../44-ceph-mds_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:41.383 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-mds (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:41.415 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package cephadm. 2026-02-12T19:05:41.423 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../45-cephadm_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:41.423 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking cephadm (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:41.440 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-12T19:05:41.448 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../46-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-12T19:05:41.448 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-12T19:05:41.470 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-12T19:05:41.478 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../47-ceph-mgr-cephadm_20.3.0-5230-gafefaa40-1jammy_all.deb ... 2026-02-12T19:05:41.478 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-mgr-cephadm (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:41.500 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-12T19:05:41.508 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../48-python3-repoze.lru_0.7-2_all.deb ... 2026-02-12T19:05:41.508 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-12T19:05:41.524 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-routes. 2026-02-12T19:05:41.532 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../49-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-12T19:05:41.532 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-12T19:05:41.552 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-12T19:05:41.561 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../50-ceph-mgr-dashboard_20.3.0-5230-gafefaa40-1jammy_all.deb ... 2026-02-12T19:05:41.561 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-mgr-dashboard (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:42.244 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-12T19:05:42.253 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../51-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-12T19:05:42.254 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-12T19:05:42.292 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-joblib. 2026-02-12T19:05:42.300 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../52-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-12T19:05:42.301 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-12T19:05:42.327 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-12T19:05:42.336 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../53-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-12T19:05:42.336 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-12T19:05:42.351 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-sklearn. 2026-02-12T19:05:42.359 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../54-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-12T19:05:42.359 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-12T19:05:42.443 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-12T19:05:42.452 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../55-ceph-mgr-diskprediction-local_20.3.0-5230-gafefaa40-1jammy_all.deb ... 2026-02-12T19:05:42.452 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-mgr-diskprediction-local (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:42.608 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-cachetools. 2026-02-12T19:05:42.616 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../56-python3-cachetools_5.0.0-1_all.deb ... 2026-02-12T19:05:42.616 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-12T19:05:42.631 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-rsa. 2026-02-12T19:05:42.640 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../57-python3-rsa_4.8-1_all.deb ... 2026-02-12T19:05:42.640 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-12T19:05:42.658 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-google-auth. 2026-02-12T19:05:42.667 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../58-python3-google-auth_1.5.1-3_all.deb ... 2026-02-12T19:05:42.667 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-12T19:05:42.684 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-12T19:05:42.692 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../59-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-12T19:05:42.692 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-12T19:05:42.709 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-websocket. 2026-02-12T19:05:42.717 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../60-python3-websocket_1.2.3-1_all.deb ... 2026-02-12T19:05:42.717 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-12T19:05:42.735 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-12T19:05:42.743 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../61-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-12T19:05:42.743 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-12T19:05:42.837 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-12T19:05:42.846 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../62-ceph-mgr-k8sevents_20.3.0-5230-gafefaa40-1jammy_all.deb ... 2026-02-12T19:05:42.846 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-mgr-k8sevents (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:42.861 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-12T19:05:42.869 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../63-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-12T19:05:42.869 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-12T19:05:42.885 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-12T19:05:42.893 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../64-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-12T19:05:42.894 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-12T19:05:42.909 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package jq. 2026-02-12T19:05:42.917 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../65-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-12T19:05:42.917 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-12T19:05:42.932 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package socat. 2026-02-12T19:05:42.940 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../66-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-12T19:05:42.940 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-12T19:05:42.960 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package xmlstarlet. 2026-02-12T19:05:42.968 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../67-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-12T19:05:42.969 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-12T19:05:43.005 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-test. 2026-02-12T19:05:43.014 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../68-ceph-test_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:43.014 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-test (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:43.647 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package ceph-volume. 2026-02-12T19:05:43.655 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../69-ceph-volume_20.3.0-5230-gafefaa40-1jammy_all.deb ... 2026-02-12T19:05:43.656 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking ceph-volume (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:43.679 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-12T19:05:43.688 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../70-libcephfs-daemon_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:43.688 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libcephfs-daemon (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:43.703 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-12T19:05:43.711 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../71-libcephfs-proxy2_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:43.711 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libcephfs-proxy2 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:43.726 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-12T19:05:43.734 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../72-libcephfs-dev_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:43.734 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking libcephfs-dev (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:43.750 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package nvme-cli. 2026-02-12T19:05:43.758 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../73-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-12T19:05:43.759 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-12T19:05:43.792 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-12T19:05:43.800 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../74-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-12T19:05:43.801 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-12T19:05:43.830 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-12T19:05:43.838 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../75-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-12T19:05:43.839 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-12T19:05:43.853 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-pluggy. 2026-02-12T19:05:43.862 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../76-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-12T19:05:43.862 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-12T19:05:43.878 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-psutil. 2026-02-12T19:05:43.886 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../77-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-12T19:05:43.887 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-12T19:05:43.906 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-py. 2026-02-12T19:05:43.914 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../78-python3-py_1.10.0-1_all.deb ... 2026-02-12T19:05:43.914 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-12T19:05:43.935 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-pygments. 2026-02-12T19:05:43.943 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../79-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-12T19:05:43.943 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-12T19:05:43.984 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-toml. 2026-02-12T19:05:43.993 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../80-python3-toml_0.10.2-1_all.deb ... 2026-02-12T19:05:43.993 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-12T19:05:44.008 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-pytest. 2026-02-12T19:05:44.016 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../81-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-12T19:05:44.016 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-12T19:05:44.038 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-simplejson. 2026-02-12T19:05:44.046 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../82-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-12T19:05:44.047 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-12T19:05:44.071 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package python3-webob. 2026-02-12T19:05:44.080 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../83-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-12T19:05:44.080 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-12T19:05:44.097 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-12T19:05:44.106 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../84-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-12T19:05:44.106 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-12T19:05:44.173 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package radosgw. 2026-02-12T19:05:44.182 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../85-radosgw_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:44.182 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking radosgw (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:44.352 INFO:teuthology.orchestra.run.trial010.stdout:Selecting previously unselected package rbd-fuse. 2026-02-12T19:05:44.360 INFO:teuthology.orchestra.run.trial010.stdout:Preparing to unpack .../86-rbd-fuse_20.3.0-5230-gafefaa40-1jammy_amd64.deb ... 2026-02-12T19:05:44.361 INFO:teuthology.orchestra.run.trial010.stdout:Unpacking rbd-fuse (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:44.388 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-12T19:05:44.453 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-12T19:05:44.454 INFO:teuthology.orchestra.run.trial010.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-12T19:05:44.525 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-12T19:05:44.787 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-12T19:05:45.144 INFO:teuthology.orchestra.run.trial010.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-12T19:05:45.154 INFO:teuthology.orchestra.run.trial010.stdout:Setting up cephadm (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:45.197 INFO:teuthology.orchestra.run.trial010.stdout:Adding system user cephadm....done 2026-02-12T19:05:45.204 INFO:teuthology.orchestra.run.trial010.stdout:usermod: no changes 2026-02-12T19:05:45.210 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-12T19:05:45.282 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-12T19:05:45.283 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-12T19:05:45.353 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-12T19:05:45.428 INFO:teuthology.orchestra.run.trial010.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-12T19:05:45.429 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-12T19:05:45.524 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-12T19:05:45.648 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-12T19:05:45.724 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-12T19:05:45.800 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-ceph-argparse (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:45.867 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-12T19:05:45.868 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-12T19:05:45.869 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-12T19:05:45.870 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-12T19:05:45.871 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-12T19:05:45.990 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-12T19:05:46.066 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libcephfs-proxy2 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:46.068 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-12T19:05:46.143 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-12T19:05:46.235 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-12T19:05:46.480 INFO:teuthology.orchestra.run.trial010.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-12T19:05:46.481 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-12T19:05:46.575 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-12T19:05:46.703 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-12T19:05:46.787 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-12T19:05:46.858 INFO:teuthology.orchestra.run.trial010.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-12T19:05:46.859 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-ceph-common (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:46.952 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-12T19:05:47.459 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-12T19:05:47.464 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-12T19:05:47.538 INFO:teuthology.orchestra.run.trial010.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-12T19:05:47.539 INFO:teuthology.orchestra.run.trial010.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-12T19:05:47.540 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-12T19:05:47.616 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-12T19:05:47.689 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-12T19:05:47.690 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-12T19:05:47.767 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-12T19:05:47.839 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-12T19:05:47.916 INFO:teuthology.orchestra.run.trial010.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-12T19:05:47.917 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-12T19:05:47.999 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-12T19:05:48.000 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-12T19:05:48.077 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-12T19:05:48.160 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-12T19:05:48.235 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-12T19:05:48.236 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-12T19:05:48.367 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-12T19:05:48.437 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-12T19:05:48.439 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-12T19:05:48.518 INFO:teuthology.orchestra.run.trial010.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-12T19:05:48.519 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-12T19:05:48.651 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-12T19:05:48.652 INFO:teuthology.orchestra.run.trial010.stdout:Setting up librados2 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:48.653 INFO:teuthology.orchestra.run.trial010.stdout:Setting up librgw2 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:48.654 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libsqlite3-mod-ceph (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:48.655 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-12T19:05:49.167 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libcephfs2 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.168 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libradosstriper1 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.169 INFO:teuthology.orchestra.run.trial010.stdout:Setting up librbd1 (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.170 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-mgr-modules-core (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.171 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-fuse (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.242 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-12T19:05:49.242 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-12T19:05:49.628 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libcephfs-dev (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.629 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-rados (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.630 INFO:teuthology.orchestra.run.trial010.stdout:Setting up libcephfs-daemon (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.631 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-rbd (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.632 INFO:teuthology.orchestra.run.trial010.stdout:Setting up rbd-fuse (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.633 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-rgw (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.634 INFO:teuthology.orchestra.run.trial010.stdout:Setting up python3-cephfs (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.635 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-common (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:49.662 INFO:teuthology.orchestra.run.trial010.stdout:Adding group ceph....done 2026-02-12T19:05:49.697 INFO:teuthology.orchestra.run.trial010.stdout:Adding system user ceph....done 2026-02-12T19:05:49.707 INFO:teuthology.orchestra.run.trial010.stdout:Setting system user ceph properties....done 2026-02-12T19:05:49.713 INFO:teuthology.orchestra.run.trial010.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-12T19:05:49.799 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-12T19:05:50.075 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-12T19:05:50.430 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-test (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:50.431 INFO:teuthology.orchestra.run.trial010.stdout:Setting up radosgw (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:50.671 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-12T19:05:50.671 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-12T19:05:51.011 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-base (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:51.130 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-12T19:05:51.469 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-mds (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:51.528 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-12T19:05:51.528 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-12T19:05:51.894 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-mgr (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:51.976 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-12T19:05:51.976 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-12T19:05:52.331 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-mgr-k8sevents (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:52.332 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-osd-classic (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:52.349 INFO:teuthology.orchestra.run.trial010.stdout:update-alternatives: using /usr/bin/ceph-osd-classic to provide /usr/bin/ceph-osd (ceph-osd) in auto mode 2026-02-12T19:05:52.358 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-mgr-diskprediction-local (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:52.369 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-mon (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:52.442 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-12T19:05:52.442 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-12T19:05:52.822 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-mgr-cephadm (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:52.833 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-mgr-dashboard (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:52.844 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-osd (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:52.926 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-12T19:05:52.926 INFO:teuthology.orchestra.run.trial010.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-12T19:05:53.303 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:53.304 INFO:teuthology.orchestra.run.trial010.stdout:Setting up ceph-volume (20.3.0-5230-gafefaa40-1jammy) ... 2026-02-12T19:05:53.413 INFO:teuthology.orchestra.run.trial010.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-12T19:05:53.498 INFO:teuthology.orchestra.run.trial010.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.11) ... 2026-02-12T19:05:53.916 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:05:53.916 INFO:teuthology.orchestra.run.trial010.stdout:Running kernel seems to be up-to-date. 2026-02-12T19:05:53.916 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:05:53.916 INFO:teuthology.orchestra.run.trial010.stdout:The processor microcode seems to be up-to-date. 2026-02-12T19:05:53.916 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:05:53.917 INFO:teuthology.orchestra.run.trial010.stdout:No services need to be restarted. 2026-02-12T19:05:53.917 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:05:53.917 INFO:teuthology.orchestra.run.trial010.stdout:No containers need to be restarted. 2026-02-12T19:05:53.917 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:05:53.917 INFO:teuthology.orchestra.run.trial010.stdout:No user sessions are running outdated binaries. 2026-02-12T19:05:53.917 INFO:teuthology.orchestra.run.trial010.stdout: 2026-02-12T19:05:53.917 INFO:teuthology.orchestra.run.trial010.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-12T19:05:54.570 INFO:teuthology.orchestra.run.trial010.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-12T19:05:54.576 DEBUG:teuthology.parallel:result is None 2026-02-12T19:05:54.577 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-12T19:05:54.759 DEBUG:teuthology.orchestra.run.trial010:> dpkg-query -W -f '${Version}' ceph 2026-02-12T19:05:54.770 INFO:teuthology.orchestra.run.trial010.stdout:20.3.0-5230-gafefaa40-1jammy 2026-02-12T19:05:54.771 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5230-gafefaa40-1jammy 2026-02-12T19:05:54.771 INFO:teuthology.task.install:The correct ceph version 20.3.0-5230-gafefaa40-1jammy is installed. 2026-02-12T19:05:54.774 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-12T19:05:54.774 DEBUG:teuthology.orchestra.run.trial010:> set -ex 2026-02-12T19:05:54.774 DEBUG:teuthology.orchestra.run.trial010:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-12T19:05:54.821 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-12T19:05:54.821 DEBUG:teuthology.orchestra.run.trial010:> set -ex 2026-02-12T19:05:54.821 DEBUG:teuthology.orchestra.run.trial010:> sudo dd of=/usr/bin/daemon-helper 2026-02-12T19:05:54.873 DEBUG:teuthology.orchestra.run.trial010:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-12T19:05:54.924 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-12T19:05:54.924 DEBUG:teuthology.orchestra.run.trial010:> set -ex 2026-02-12T19:05:54.925 DEBUG:teuthology.orchestra.run.trial010:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-12T19:05:54.973 DEBUG:teuthology.orchestra.run.trial010:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-12T19:05:55.024 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-12T19:05:55.024 DEBUG:teuthology.orchestra.run.trial010:> set -ex 2026-02-12T19:05:55.024 DEBUG:teuthology.orchestra.run.trial010:> sudo dd of=/usr/bin/stdin-killer 2026-02-12T19:05:55.073 DEBUG:teuthology.orchestra.run.trial010:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-12T19:05:55.124 INFO:teuthology.run_tasks:Running task workunit... 2026-02-12T19:05:55.151 INFO:tasks.workunit:Pulling workunits from ref afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-12T19:05:55.151 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-12T19:05:55.151 INFO:tasks.workunit:timeout=3h 2026-02-12T19:05:55.152 INFO:tasks.workunit:cleanup=True 2026-02-12T19:05:55.152 DEBUG:teuthology.orchestra.run.trial010:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-12T19:05:55.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T19:05:55.169 INFO:teuthology.orchestra.run.trial010.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-12T19:05:55.169 DEBUG:teuthology.orchestra.run.trial010:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-12T19:05:55.216 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-12T19:05:55.217 DEBUG:teuthology.orchestra.run.trial010:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-12T19:05:55.261 DEBUG:teuthology.orchestra.run.trial010:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout afefaa40503a4bf024e96f97dd7f852b75f00ad6 2026-02-12T19:05:55.311 INFO:tasks.workunit.client.0.trial010.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-12T19:06:16.798 INFO:tasks.workunit.client.0.trial010.stderr:Note: switching to 'afefaa40503a4bf024e96f97dd7f852b75f00ad6'. 2026-02-12T19:06:16.798 INFO:tasks.workunit.client.0.trial010.stderr: 2026-02-12T19:06:16.798 INFO:tasks.workunit.client.0.trial010.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-12T19:06:16.799 INFO:tasks.workunit.client.0.trial010.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-12T19:06:16.799 INFO:tasks.workunit.client.0.trial010.stderr:state without impacting any branches by switching back to a branch. 2026-02-12T19:06:16.799 INFO:tasks.workunit.client.0.trial010.stderr: 2026-02-12T19:06:16.799 INFO:tasks.workunit.client.0.trial010.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-12T19:06:16.799 INFO:tasks.workunit.client.0.trial010.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-12T19:06:16.799 INFO:tasks.workunit.client.0.trial010.stderr: 2026-02-12T19:06:16.799 INFO:tasks.workunit.client.0.trial010.stderr: git switch -c 2026-02-12T19:06:16.799 INFO:tasks.workunit.client.0.trial010.stderr: 2026-02-12T19:06:16.799 INFO:tasks.workunit.client.0.trial010.stderr:Or undo this operation with: 2026-02-12T19:06:16.799 INFO:tasks.workunit.client.0.trial010.stderr: 2026-02-12T19:06:16.800 INFO:tasks.workunit.client.0.trial010.stderr: git switch - 2026-02-12T19:06:16.800 INFO:tasks.workunit.client.0.trial010.stderr: 2026-02-12T19:06:16.800 INFO:tasks.workunit.client.0.trial010.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-12T19:06:16.800 INFO:tasks.workunit.client.0.trial010.stderr: 2026-02-12T19:06:16.800 INFO:tasks.workunit.client.0.trial010.stderr:HEAD is now at afefaa40503 Merge branch 'block_non_4k_chunks' of https://github.com/aainscow/ceph into wip-yuri4-testing-2026-02-11-2126 2026-02-12T19:06:16.808 DEBUG:teuthology.orchestra.run.trial010:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-12T19:06:16.854 DEBUG:teuthology.orchestra.run.trial010:> set -ex 2026-02-12T19:06:16.855 DEBUG:teuthology.orchestra.run.trial010:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-12T19:06:16.900 INFO:tasks.workunit:Running workunits matching misc on client.0... 2026-02-12T19:06:16.900 INFO:tasks.workunit:Running workunit misc/mclock-config.sh... 2026-02-12T19:06:16.901 DEBUG:teuthology.orchestra.run.trial010:workunit test misc/mclock-config.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh 2026-02-12T19:06:16.951 INFO:tasks.workunit.client.0.trial010.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-12T19:06:16.956 INFO:tasks.workunit.client.0.trial010.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-12T19:06:16.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-12T19:06:16.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-12T19:06:16.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-12T19:06:16.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-12T19:06:16.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-12T19:06:16.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-12T19:06:16.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-12T19:06:16.957 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-12T19:06:16.957 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/mclock-config 2026-02-12T19:06:16.957 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:21: run: local dir=td/mclock-config 2026-02-12T19:06:16.957 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:22: run: shift 2026-02-12T19:06:16.957 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: export CEPH_MON=127.0.0.1:7124 2026-02-12T19:06:16.957 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:24: run: CEPH_MON=127.0.0.1:7124 2026-02-12T19:06:16.957 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:25: run: export CEPH_ARGS 2026-02-12T19:06:16.957 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: uuidgen 2026-02-12T19:06:16.958 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:26: run: CEPH_ARGS+='--fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none ' 2026-02-12T19:06:16.958 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:27: run: CEPH_ARGS+='--mon-host=127.0.0.1:7124 ' 2026-02-12T19:06:16.958 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:28: run: CEPH_ARGS+='--debug-mclock 20 ' 2026-02-12T19:06:16.958 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: set 2026-02-12T19:06:16.958 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-12T19:06:16.960 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:30: run: local 'funcs=TEST_backfill_limit_adjustment_mclock 2026-02-12T19:06:16.960 INFO:tasks.workunit.client.0.trial010.stderr:TEST_profile_builtin_to_custom 2026-02-12T19:06:16.960 INFO:tasks.workunit.client.0.trial010.stderr:TEST_profile_custom_to_builtin 2026-02-12T19:06:16.960 INFO:tasks.workunit.client.0.trial010.stderr:TEST_recovery_limit_adjustment_mclock' 2026-02-12T19:06:16.960 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-12T19:06:16.960 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-12T19:06:16.960 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-12T19:06:16.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-12T19:06:16.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:16.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:16.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:16.961 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:16.961 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:16.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:16.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:16.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:16.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:16.962 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:16.962 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:16.963 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:16.964 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:16.964 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:16.964 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:16.965 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:16.965 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:16.966 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:16.966 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:16.967 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:16.968 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:16.968 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:16.969 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:16.969 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:16.969 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:16.969 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:16.970 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:16.970 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:16.971 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-12T19:06:16.972 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:06:16.972 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:16.972 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:16.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7506 2026-02-12T19:06:16.973 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:06:16.975 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-12T19:06:16.975 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-12T19:06:16.975 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_backfill_limit_adjustment_mclock td/mclock-config 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:236: TEST_backfill_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:238: TEST_backfill_limit_adjustment_mclock: setup td/mclock-config 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:16.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:16.977 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:16.977 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:16.977 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:16.977 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:16.978 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:16.979 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:16.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:16.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:16.980 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:16.981 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:16.981 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:16.982 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:16.982 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:16.983 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:16.984 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:16.984 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:16.985 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:16.985 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:16.985 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:16.985 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:16.986 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:16.986 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:16.986 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-12T19:06:16.988 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:06:16.988 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:16.988 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:16.988 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7506 2026-02-12T19:06:16.989 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:06:16.990 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:06:16.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:06:16.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-12T19:06:16.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:239: TEST_backfill_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-12T19:06:16.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-12T19:06:16.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:06:16.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:06:16.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:06:16.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-12T19:06:16.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-12T19:06:17.023 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:06:17.023 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:17.023 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:17.024 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:17.024 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:17.024 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:17.024 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:17.024 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:06:17.053 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:06:17.054 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:06:17.054 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:06:17.054 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:06:17.054 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:06:17.055 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:06:17.055 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:06:17.056 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:06:17.056 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:06:17.056 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:17.056 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:17.056 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:17.056 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-mon.a.asok 2026-02-12T19:06:17.056 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:06:17.056 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7506/ceph-mon.a.asok config get fsid 2026-02-12T19:06:17.111 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:06:17.111 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:06:17.112 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:06:17.112 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:06:17.113 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:06:17.113 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:06:17.113 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:06:17.113 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:06:17.113 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:17.113 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:17.113 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:17.114 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-mon.a.asok 2026-02-12T19:06:17.114 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:06:17.114 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7506/ceph-mon.a.asok config get mon_host 2026-02-12T19:06:17.169 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:240: TEST_backfill_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-12T19:06:17.169 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-12T19:06:17.169 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:06:17.169 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:06:17.169 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:06:17.169 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-12T19:06:17.169 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:06:17.281 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:06:17.281 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:17.282 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:17.282 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:17.282 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:17.282 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:17.282 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:17.282 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:06:17.284 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:06:17.306 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:242: TEST_backfill_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-12T19:06:17.306 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-12T19:06:17.306 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:06:17.306 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:06:17.306 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:06:17.306 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-12T19:06:17.306 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-12T19:06:17.306 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:17.307 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:06:17.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:06:17.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-12T19:06:17.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-12T19:06:17.309 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:06:17.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=67f95237-1a74-4645-97e4-332b3a5cc53d 2026-02-12T19:06:17.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 67f95237-1a74-4645-97e4-332b3a5cc53d' 2026-02-12T19:06:17.310 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 67f95237-1a74-4645-97e4-332b3a5cc53d 2026-02-12T19:06:17.310 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:06:17.322 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCpJI5pBaDgEhAAKE3Tvgg9JCsw7xcTLCLTww== 2026-02-12T19:06:17.322 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCpJI5pBaDgEhAAKE3Tvgg9JCsw7xcTLCLTww=="}' 2026-02-12T19:06:17.322 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 67f95237-1a74-4645-97e4-332b3a5cc53d -i td/mclock-config/0/new.json 2026-02-12T19:06:17.431 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:17.439 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-12T19:06:17.440 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQCpJI5pBaDgEhAAKE3Tvgg9JCsw7xcTLCLTww== --osd-uuid 67f95237-1a74-4645-97e4-332b3a5cc53d 2026-02-12T19:06:17.455 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:17.446+0000 7fbfade87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:17.457 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:17.450+0000 7fbfade87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:17.458 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:17.450+0000 7fbfade87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:17.459 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:17.450+0000 7fbfade87a40 -1 bdev(0x55808d518400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:06:17.459 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:17.450+0000 7fbfade87a40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-12T19:06:18.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-12T19:06:18.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:06:18.112 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:06:18.112 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:06:18.112 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:06:18.230 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:06:18.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:06:18.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-12T19:06:18.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:06:18.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:06:18.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:06:18.254 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:18.246+0000 7f2118154a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:18.256 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:18.246+0000 7f2118154a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:18.257 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:18.250+0000 7f2118154a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:18.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:06:18.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:06:18.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:06:18.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:06:18.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:06:18.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:18.338 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:18.339 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:06:18.339 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:18.339 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:18.450 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:18.671 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:18.662+0000 7f2118154a40 -1 Falling back to public interface 2026-02-12T19:06:19.332 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:19.322+0000 7f2118154a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:06:19.451 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:06:19.451 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:19.452 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:06:19.452 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:06:19.452 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:19.453 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:19.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:20.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:06:20.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:20.563 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:06:20.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:06:20.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:20.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:20.768 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2041539504,v1:127.0.0.1:6801/2041539504] [v2:127.0.0.1:6802/2041539504,v1:127.0.0.1:6803/2041539504] exists,up 67f95237-1a74-4645-97e4-332b3a5cc53d 2026-02-12T19:06:20.769 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:06:20.769 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:06:20.769 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:06:20.769 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-12T19:06:20.769 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: bc 2026-02-12T19:06:20.770 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:20.770 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:20.770 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:20.771 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:20.771 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:20.771 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:20.771 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:20.771 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:20.771 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_max_backfills 2026-02-12T19:06:20.834 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:244: TEST_backfill_limit_adjustment_mclock: local backfills=1 2026-02-12T19:06:20.834 INFO:tasks.workunit.client.0.trial010.stdout:osd_max_backfills: 1 2026-02-12T19:06:20.834 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:246: TEST_backfill_limit_adjustment_mclock: echo 'osd_max_backfills: 1' 2026-02-12T19:06:20.834 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:251: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-12T19:06:21.045 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:252: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-12T19:06:23.048 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-12T19:06:23.048 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: bc 2026-02-12T19:06:23.048 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:23.048 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:23.048 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:23.049 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:23.049 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:23.049 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:23.049 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:23.049 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:23.050 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_max_backfills 2026-02-12T19:06:23.110 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:254: TEST_backfill_limit_adjustment_mclock: local max_backfills=1 2026-02-12T19:06:23.110 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:255: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-12T19:06:23.110 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-12T19:06:23.110 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: bc 2026-02-12T19:06:23.111 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:23.111 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:23.111 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:23.111 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:23.111 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:23.111 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:23.112 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:23.112 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:23.112 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok dump_recovery_reservations 2026-02-12T19:06:23.170 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:259: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-12T19:06:23.170 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:260: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-12T19:06:23.171 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-12T19:06:23.171 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: bc 2026-02-12T19:06:23.172 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:23.172 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:23.172 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:23.172 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:23.172 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:23.172 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:23.172 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:23.173 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:23.173 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok dump_recovery_reservations 2026-02-12T19:06:23.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:262: TEST_backfill_limit_adjustment_mclock: max_backfills=1 2026-02-12T19:06:23.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:263: TEST_backfill_limit_adjustment_mclock: test 1 = 1 2026-02-12T19:06:23.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:267: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-12T19:06:23.447 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:268: TEST_backfill_limit_adjustment_mclock: ceph config set osd.0 osd_max_backfills 20 2026-02-12T19:06:23.656 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:269: TEST_backfill_limit_adjustment_mclock: sleep 2 2026-02-12T19:06:25.659 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-12T19:06:25.659 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: bc 2026-02-12T19:06:25.660 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:25.660 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:25.661 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:25.661 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:25.661 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:25.661 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:25.661 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:25.661 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:25.662 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_max_backfills 2026-02-12T19:06:25.722 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:271: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-12T19:06:25.722 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:272: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-12T19:06:25.724 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-12T19:06:25.724 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: bc 2026-02-12T19:06:25.724 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:25.724 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:25.724 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:25.724 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:25.724 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:25.724 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:25.724 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:25.725 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:25.725 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok dump_recovery_reservations 2026-02-12T19:06:25.783 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:276: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-12T19:06:25.783 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:277: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-12T19:06:25.785 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-12T19:06:25.785 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: bc 2026-02-12T19:06:25.785 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:25.785 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:25.785 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:25.785 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:25.786 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:25.786 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:25.786 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:25.786 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:25.787 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok dump_recovery_reservations 2026-02-12T19:06:25.845 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:279: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-12T19:06:25.846 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:280: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-12T19:06:25.846 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:284: TEST_backfill_limit_adjustment_mclock: kill_daemons td/mclock-config TERM osd 2026-02-12T19:06:25.846 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:25.846 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:25.846 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:25.846 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:25.846 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:26.152 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:26.152 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:285: TEST_backfill_limit_adjustment_mclock: ceph osd down 0 2026-02-12T19:06:26.363 INFO:tasks.workunit.client.0.trial010.stderr:osd.0 is already down. 2026-02-12T19:06:26.372 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:286: TEST_backfill_limit_adjustment_mclock: wait_for_osd down 0 2026-02-12T19:06:26.372 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-12T19:06:26.373 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:06:26.373 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:06:26.373 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:06:26.373 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:26.373 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:06:26.373 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:26.374 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:26.374 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-12T19:06:26.600 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/2041539504,v1:127.0.0.1:6801/2041539504] [v2:127.0.0.1:6802/2041539504,v1:127.0.0.1:6803/2041539504] exists 67f95237-1a74-4645-97e4-332b3a5cc53d 2026-02-12T19:06:26.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:06:26.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:06:26.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:06:26.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:287: TEST_backfill_limit_adjustment_mclock: activate_osd td/mclock-config 0 --osd-op-queue=mclock_scheduler 2026-02-12T19:06:26.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/mclock-config 2026-02-12T19:06:26.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T19:06:26.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/mclock-config/0 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-12T19:06:26.604 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:06:26.605 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:06:26.606 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:06:26.606 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T19:06:26.606 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=--osd-op-queue=mclock_scheduler 2026-02-12T19:06:26.606 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/mclock-config/0 2026-02-12T19:06:26.606 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:06:26.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-12T19:06:26.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --debug-osd=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler 2026-02-12T19:06:26.607 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/mclock-config/0/whoami 2026-02-12T19:06:26.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-12T19:06:26.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T19:06:26.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T19:06:26.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T19:06:26.627 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:26.618+0000 7f3e5a200a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:26.628 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:26.618+0000 7f3e5a200a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:26.629 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:26.622+0000 7f3e5a200a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:26.826 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-12T19:06:26.827 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:06:26.827 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:06:26.827 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:06:26.827 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:06:26.827 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:26.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:26.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:06:26.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:26.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:27.031 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:27.022+0000 7f3e5a200a40 -1 Falling back to public interface 2026-02-12T19:06:27.042 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:27.736 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:27.726+0000 7f3e5a200a40 -1 osd.0 6 log_to_monitors true 2026-02-12T19:06:28.044 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:06:28.044 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:28.044 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:06:28.045 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:06:28.045 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:28.045 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:28.259 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 9 up_thru 0 down_at 6 last_clean_interval [5,6) [v2:127.0.0.1:6800/1300116791,v1:127.0.0.1:6801/1300116791] [v2:127.0.0.1:6802/1300116791,v1:127.0.0.1:6803/1300116791] exists,up 67f95237-1a74-4645-97e4-332b3a5cc53d 2026-02-12T19:06:28.260 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:06:28.260 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:06:28.260 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:06:28.260 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: jq .osd_max_backfills 2026-02-12T19:06:28.260 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: bc 2026-02-12T19:06:28.261 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:28.261 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:28.261 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:28.261 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:28.261 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.261 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.261 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:28.262 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:28.262 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_max_backfills 2026-02-12T19:06:28.323 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:290: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-12T19:06:28.323 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:291: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-12T19:06:28.324 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: jq .local_reservations.max_allowed 2026-02-12T19:06:28.324 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: bc 2026-02-12T19:06:28.324 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:28.324 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:28.325 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:28.325 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:28.325 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.325 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.325 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:28.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:28.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok dump_recovery_reservations 2026-02-12T19:06:28.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:295: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-12T19:06:28.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:296: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-12T19:06:28.384 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: jq .remote_reservations.max_allowed 2026-02-12T19:06:28.384 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: bc 2026-02-12T19:06:28.384 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:28.384 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:28.385 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:28.385 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:28.386 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.386 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.386 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:28.386 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:28.386 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok dump_recovery_reservations 2026-02-12T19:06:28.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:298: TEST_backfill_limit_adjustment_mclock: max_backfills=20 2026-02-12T19:06:28.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:299: TEST_backfill_limit_adjustment_mclock: test 20 = 20 2026-02-12T19:06:28.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:301: TEST_backfill_limit_adjustment_mclock: teardown td/mclock-config 2026-02-12T19:06:28.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:28.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:28.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:28.444 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:28.444 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:28.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:28.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:28.445 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:28.551 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:28.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:28.552 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:28.553 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:28.553 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:28.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:28.554 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:28.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:28.555 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:28.555 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:28.555 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:28.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:28.557 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:28.557 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:28.562 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:28.562 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.562 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:28.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:28.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:28.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-12T19:06:28.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:28.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:28.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:28.565 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:28.565 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:28.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:28.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:28.566 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:28.566 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:28.567 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:28.568 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:28.568 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:28.569 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:28.569 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:28.570 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:28.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:28.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:28.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:28.572 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:28.572 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:28.573 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:28.573 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:28.574 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:28.574 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.574 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:28.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:28.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:28.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-12T19:06:28.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-12T19:06:28.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-12T19:06:28.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-12T19:06:28.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:28.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:28.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:28.576 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:28.576 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:28.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:28.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:28.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:28.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:28.579 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:28.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:28.581 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:28.582 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:28.582 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:28.583 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:28.584 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:28.584 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:28.584 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:28.585 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:28.585 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:28.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:28.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:28.588 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:28.588 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.588 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.588 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:28.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:28.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:28.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-12T19:06:28.590 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:06:28.591 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.591 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.591 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7506 2026-02-12T19:06:28.592 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:06:28.592 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:06:28.592 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:06:28.592 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-12T19:06:28.592 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_builtin_to_custom td/mclock-config 2026-02-12T19:06:28.592 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:39: TEST_profile_builtin_to_custom: local dir=td/mclock-config 2026-02-12T19:06:28.592 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:41: TEST_profile_builtin_to_custom: run_mon td/mclock-config a 2026-02-12T19:06:28.592 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-12T19:06:28.592 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:06:28.592 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:06:28.593 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:06:28.593 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-12T19:06:28.593 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-12T19:06:28.612 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:06:28.612 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:28.612 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:28.612 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:28.612 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.612 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.613 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:28.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:06:28.640 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:06:28.641 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:06:28.641 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:06:28.641 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:06:28.641 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:06:28.641 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:06:28.641 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:06:28.641 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:06:28.642 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:06:28.642 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:28.642 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.642 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.642 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-mon.a.asok 2026-02-12T19:06:28.642 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:06:28.642 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7506/ceph-mon.a.asok config get fsid 2026-02-12T19:06:28.699 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:06:28.699 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:06:28.700 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:06:28.700 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:06:28.700 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:06:28.701 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:06:28.701 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:06:28.701 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:06:28.701 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:28.701 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.701 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.702 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-mon.a.asok 2026-02-12T19:06:28.702 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:06:28.702 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7506/ceph-mon.a.asok config get mon_host 2026-02-12T19:06:28.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:42: TEST_profile_builtin_to_custom: run_mgr td/mclock-config x 2026-02-12T19:06:28.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-12T19:06:28.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:06:28.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:06:28.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:06:28.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-12T19:06:28.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:06:28.869 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:06:28.869 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:28.869 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:28.870 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:28.870 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.870 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.870 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:28.870 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:06:28.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:06:28.891 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:43: TEST_profile_builtin_to_custom: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-12T19:06:28.891 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-12T19:06:28.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:06:28.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:06:28.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:06:28.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-12T19:06:28.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-12T19:06:28.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:06:28.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:06:28.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:06:28.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-12T19:06:28.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-12T19:06:28.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:06:28.893 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:06:28.893 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-12T19:06:28.893 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:06:28.893 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:28.893 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:28.893 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:28.893 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:28.893 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:28.893 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:28.893 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:28.893 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:06:28.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:06:28.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:06:28.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-12T19:06:28.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-12T19:06:28.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:06:28.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:06:28.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:06:28.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:06:28.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:06:28.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-12T19:06:28.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-12T19:06:28.895 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:06:28.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2d677ecb-bd4b-4bac-94f2-70ecd22c3a5c 2026-02-12T19:06:28.896 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 2d677ecb-bd4b-4bac-94f2-70ecd22c3a5c 2026-02-12T19:06:28.896 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2d677ecb-bd4b-4bac-94f2-70ecd22c3a5c' 2026-02-12T19:06:28.896 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:06:28.903 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC0JI5pbIGJNRAAS16dhZ+ROY1TLL5Zrxpe8w== 2026-02-12T19:06:28.903 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC0JI5pbIGJNRAAS16dhZ+ROY1TLL5Zrxpe8w=="}' 2026-02-12T19:06:28.903 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2d677ecb-bd4b-4bac-94f2-70ecd22c3a5c -i td/mclock-config/0/new.json 2026-02-12T19:06:29.014 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:29.021 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-12T19:06:29.023 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQC0JI5pbIGJNRAAS16dhZ+ROY1TLL5Zrxpe8w== --osd-uuid 2d677ecb-bd4b-4bac-94f2-70ecd22c3a5c 2026-02-12T19:06:29.038 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:29.030+0000 7f69d0724a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:29.040 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:29.030+0000 7f69d0724a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:29.041 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:29.030+0000 7f69d0724a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:29.041 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:29.030+0000 7f69d0724a40 -1 bdev(0x5607b8e00400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:06:29.041 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:29.030+0000 7f69d0724a40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-12T19:06:29.708 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-12T19:06:29.708 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:06:29.709 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:06:29.710 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:06:29.710 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:06:29.818 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:06:29.819 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:06:29.819 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-12T19:06:29.819 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:06:29.819 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:06:29.819 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:06:29.834 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:29.826+0000 7fd61a057a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:29.835 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:29.826+0000 7fd61a057a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:29.836 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:29.826+0000 7fd61a057a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:29.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:06:29.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:06:29.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:06:29.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:06:29.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:06:29.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:29.932 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:29.932 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:06:29.932 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:29.932 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:30.038 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:30.243 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:30.234+0000 7fd61a057a40 -1 Falling back to public interface 2026-02-12T19:06:30.883 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:30.874+0000 7fd61a057a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:06:31.040 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:06:31.040 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:31.040 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:06:31.041 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:06:31.041 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:31.043 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:31.191 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:32.192 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:06:32.192 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:32.192 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:06:32.192 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:06:32.192 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:32.193 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:32.409 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1128844356,v1:127.0.0.1:6801/1128844356] [v2:127.0.0.1:6802/1128844356,v1:127.0.0.1:6803/1128844356] exists,up 2d677ecb-bd4b-4bac-94f2-70ecd22c3a5c 2026-02-12T19:06:32.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:06:32.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:06:32.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:06:32.410 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_profile 2026-02-12T19:06:32.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:46: TEST_profile_builtin_to_custom: local mclock_profile=balanced 2026-02-12T19:06:32.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:47: TEST_profile_builtin_to_custom: test balanced = balanced 2026-02-12T19:06:32.627 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-12T19:06:32.628 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-12T19:06:32.628 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:32.628 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:32.629 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:32.629 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:32.629 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:32.629 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:32.629 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-12T19:06:32.629 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_mclock_profile 2026-02-12T19:06:32.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:52: TEST_profile_builtin_to_custom: mclock_profile='"high_recovery_ops"' 2026-02-12T19:06:32.689 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: eval echo '"high_recovery_ops"' 2026-02-12T19:06:32.689 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: echo high_recovery_ops 2026-02-12T19:06:32.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:53: TEST_profile_builtin_to_custom: mclock_profile=high_recovery_ops 2026-02-12T19:06:32.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:54: TEST_profile_builtin_to_custom: test high_recovery_ops = high_recovery_ops 2026-02-12T19:06:32.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:57: TEST_profile_builtin_to_custom: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-12T19:06:32.792 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:06:32.793 INFO:tasks.workunit.client.0.trial010.stdout: "success": "" 2026-02-12T19:06:32.793 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:06:32.800 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: jq .osd_mclock_profile 2026-02-12T19:06:32.801 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-12T19:06:32.802 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:32.802 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:32.802 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:32.802 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:32.802 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:32.802 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:32.802 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-12T19:06:32.803 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_mclock_profile 2026-02-12T19:06:32.860 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:61: TEST_profile_builtin_to_custom: mclock_profile='"custom"' 2026-02-12T19:06:32.861 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: eval echo '"custom"' 2026-02-12T19:06:32.861 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: echo custom 2026-02-12T19:06:32.861 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:62: TEST_profile_builtin_to_custom: mclock_profile=custom 2026-02-12T19:06:32.861 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:63: TEST_profile_builtin_to_custom: test custom = custom 2026-02-12T19:06:32.862 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-12T19:06:32.862 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: bc 2026-02-12T19:06:32.863 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-12T19:06:32.863 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:32.863 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:32.864 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:32.864 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:32.864 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:32.864 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:32.864 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-12T19:06:32.864 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-12T19:06:32.925 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:68: TEST_profile_builtin_to_custom: local client_res=0.000000 2026-02-12T19:06:32.925 INFO:tasks.workunit.client.0.trial010.stdout:client_res = 0.000000 2026-02-12T19:06:32.926 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:69: TEST_profile_builtin_to_custom: echo 'client_res = 0.000000' 2026-02-12T19:06:32.926 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: echo '0.000000 + 0.1' 2026-02-12T19:06:32.926 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: bc -l 2026-02-12T19:06:32.927 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:70: TEST_profile_builtin_to_custom: local client_res_new=.100000 2026-02-12T19:06:32.927 INFO:tasks.workunit.client.0.trial010.stdout:client_res_new = .100000 2026-02-12T19:06:32.928 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:71: TEST_profile_builtin_to_custom: echo 'client_res_new = .100000' 2026-02-12T19:06:32.928 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:72: TEST_profile_builtin_to_custom: ceph config set osd.0 osd_mclock_scheduler_client_res .100000 2026-02-12T19:06:33.151 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-12T19:06:33.379 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:77: TEST_profile_builtin_to_custom: local res=0.100000 2026-02-12T19:06:33.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: echo '0.100000 != .100000' 2026-02-12T19:06:33.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: bc -l 2026-02-12T19:06:33.381 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:78: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-12T19:06:33.382 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: jq .osd_mclock_scheduler_client_res 2026-02-12T19:06:33.382 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: bc 2026-02-12T19:06:33.382 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: get_asok_path osd.0 2026-02-12T19:06:33.383 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:33.383 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:33.383 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:33.384 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.384 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.384 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:33.384 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: CEPH_ARGS= 2026-02-12T19:06:33.384 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-12T19:06:33.443 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:84: TEST_profile_builtin_to_custom: res=0.100000 2026-02-12T19:06:33.444 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: echo '0.100000 != .100000' 2026-02-12T19:06:33.444 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: bc -l 2026-02-12T19:06:33.446 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:85: TEST_profile_builtin_to_custom: (( 0 )) 2026-02-12T19:06:33.447 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:89: TEST_profile_builtin_to_custom: teardown td/mclock-config 2026-02-12T19:06:33.447 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:33.447 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:33.447 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:33.447 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:33.447 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:33.448 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:33.448 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:33.449 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:33.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:33.554 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:33.555 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:33.555 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:33.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:33.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:33.557 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:33.558 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:33.558 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:33.558 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:33.558 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:33.559 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:33.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:33.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:33.565 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:33.565 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.565 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:33.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:33.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:33.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-12T19:06:33.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:33.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:33.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:33.567 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:33.567 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:33.568 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:33.568 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:33.568 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:33.569 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:33.570 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:33.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:33.571 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:33.572 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:33.572 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:33.572 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:33.573 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:33.574 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:33.574 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:33.574 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:33.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:33.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:33.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:33.578 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:33.578 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.578 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:33.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:33.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:33.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-12T19:06:33.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-12T19:06:33.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-12T19:06:33.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-12T19:06:33.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:33.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:33.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:33.580 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:33.580 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:33.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:33.581 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:33.581 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:33.582 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:33.582 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:33.583 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:33.583 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:33.584 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:33.584 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:33.584 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:33.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:33.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:33.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:33.586 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:33.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:33.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:33.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:33.590 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:33.590 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.590 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.590 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:33.591 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:33.591 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:33.591 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-12T19:06:33.593 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:06:33.593 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.593 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.593 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7506 2026-02-12T19:06:33.594 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:06:33.596 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:06:33.596 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:06:33.596 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-12T19:06:33.596 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_profile_custom_to_builtin td/mclock-config 2026-02-12T19:06:33.596 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:93: TEST_profile_custom_to_builtin: local dir=td/mclock-config 2026-02-12T19:06:33.596 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:95: TEST_profile_custom_to_builtin: setup td/mclock-config 2026-02-12T19:06:33.596 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-12T19:06:33.596 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-12T19:06:33.596 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:33.597 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:33.597 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:33.597 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:33.597 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:33.597 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:33.597 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:33.597 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:33.598 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:33.598 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:33.599 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:33.599 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:33.600 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:33.600 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:33.600 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:33.601 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:33.602 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:33.602 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:33.602 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:33.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:33.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:33.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:33.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:33.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.606 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.606 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:33.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:33.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:33.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-12T19:06:33.608 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:06:33.608 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.609 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7506 2026-02-12T19:06:33.610 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:06:33.611 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:06:33.611 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:06:33.611 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-12T19:06:33.611 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:96: TEST_profile_custom_to_builtin: run_mon td/mclock-config a 2026-02-12T19:06:33.611 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-12T19:06:33.612 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:06:33.612 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:06:33.612 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:06:33.612 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-12T19:06:33.612 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-12T19:06:33.632 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:06:33.632 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:33.632 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:33.633 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:33.633 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.633 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.634 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:33.634 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:06:33.660 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:06:33.661 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:06:33.661 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:06:33.661 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:06:33.661 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:06:33.661 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:06:33.662 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:06:33.662 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:06:33.663 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:06:33.663 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:33.663 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.663 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.663 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-mon.a.asok 2026-02-12T19:06:33.663 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:06:33.663 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7506/ceph-mon.a.asok config get fsid 2026-02-12T19:06:33.720 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:06:33.720 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:06:33.720 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:06:33.720 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:06:33.720 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:06:33.721 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:06:33.722 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:06:33.722 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:06:33.722 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:33.722 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.722 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.722 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-mon.a.asok 2026-02-12T19:06:33.722 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:06:33.722 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7506/ceph-mon.a.asok config get mon_host 2026-02-12T19:06:33.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:97: TEST_profile_custom_to_builtin: run_mgr td/mclock-config x 2026-02-12T19:06:33.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-12T19:06:33.781 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:06:33.781 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:06:33.781 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:06:33.781 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-12T19:06:33.781 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:06:33.893 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:06:33.894 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:33.894 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:33.894 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:33.894 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.894 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.895 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:33.895 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:06:33.896 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:06:33.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:98: TEST_profile_custom_to_builtin: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-12T19:06:33.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-12T19:06:33.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:06:33.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-12T19:06:33.917 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:06:33.918 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:33.918 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:33.918 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:33.918 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:33.918 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:33.918 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:33.918 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:33.918 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:06:33.918 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:06:33.918 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:06:33.918 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-12T19:06:33.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-12T19:06:33.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:06:33.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:06:33.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:06:33.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:06:33.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:06:33.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-12T19:06:33.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-12T19:06:33.919 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:06:33.920 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f4e7fcfa-69ec-48a4-92c9-8b1a5b1e0e4b 2026-02-12T19:06:33.920 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 f4e7fcfa-69ec-48a4-92c9-8b1a5b1e0e4b 2026-02-12T19:06:33.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f4e7fcfa-69ec-48a4-92c9-8b1a5b1e0e4b' 2026-02-12T19:06:33.921 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:06:33.934 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC5JI5p2+VTNxAA9KyDzny6LGnGWin8g6lSlQ== 2026-02-12T19:06:33.934 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC5JI5p2+VTNxAA9KyDzny6LGnGWin8g6lSlQ=="}' 2026-02-12T19:06:33.934 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f4e7fcfa-69ec-48a4-92c9-8b1a5b1e0e4b -i td/mclock-config/0/new.json 2026-02-12T19:06:34.043 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:34.051 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-12T19:06:34.052 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQC5JI5p2+VTNxAA9KyDzny6LGnGWin8g6lSlQ== --osd-uuid f4e7fcfa-69ec-48a4-92c9-8b1a5b1e0e4b 2026-02-12T19:06:34.067 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:34.058+0000 7fa751526a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:34.069 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:34.062+0000 7fa751526a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:34.070 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:34.062+0000 7fa751526a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:34.071 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:34.062+0000 7fa751526a40 -1 bdev(0x55a8cd8fe400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:06:34.071 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:34.062+0000 7fa751526a40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-12T19:06:34.744 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-12T19:06:34.745 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:06:34.745 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:06:34.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:06:34.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:06:34.867 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:06:34.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:06:34.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-12T19:06:34.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:06:34.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:06:34.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:06:34.890 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:34.882+0000 7f476ca3aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:34.892 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:34.882+0000 7f476ca3aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:34.893 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:34.886+0000 7f476ca3aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:34.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:06:34.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:06:34.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:06:34.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:06:34.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:06:34.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:34.980 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:34.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:06:34.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:34.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:35.087 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:35.319 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:35.310+0000 7f476ca3aa40 -1 Falling back to public interface 2026-02-12T19:06:36.088 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:06:36.088 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:36.088 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:06:36.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:06:36.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:36.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:36.202 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:36.250 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:36.242+0000 7f476ca3aa40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:06:37.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:06:37.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:37.205 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:06:37.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:06:37.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:37.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:37.420 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3158929135,v1:127.0.0.1:6801/3158929135] [v2:127.0.0.1:6802/3158929135,v1:127.0.0.1:6803/3158929135] exists,up f4e7fcfa-69ec-48a4-92c9-8b1a5b1e0e4b 2026-02-12T19:06:37.421 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:06:37.421 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:06:37.421 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:06:37.421 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:101: TEST_profile_custom_to_builtin: local def_mclock_profile 2026-02-12T19:06:37.421 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_profile 2026-02-12T19:06:37.637 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:102: TEST_profile_custom_to_builtin: def_mclock_profile=balanced 2026-02-12T19:06:37.637 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:103: TEST_profile_custom_to_builtin: test balanced = balanced 2026-02-12T19:06:37.638 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-12T19:06:37.638 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-12T19:06:37.638 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:37.638 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:37.639 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:37.639 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:37.640 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:37.640 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:37.640 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-12T19:06:37.640 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_mclock_profile 2026-02-12T19:06:37.699 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:108: TEST_profile_custom_to_builtin: local 'orig_mclock_profile="high_recovery_ops"' 2026-02-12T19:06:37.700 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: eval echo '"high_recovery_ops"' 2026-02-12T19:06:37.700 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: echo high_recovery_ops 2026-02-12T19:06:37.700 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:109: TEST_profile_custom_to_builtin: orig_mclock_profile=high_recovery_ops 2026-02-12T19:06:37.700 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:110: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-12T19:06:37.700 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:113: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile custom 2026-02-12T19:06:37.791 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:06:37.791 INFO:tasks.workunit.client.0.trial010.stdout: "success": "" 2026-02-12T19:06:37.791 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:06:37.799 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-12T19:06:37.800 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-12T19:06:37.800 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:37.800 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:37.800 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:37.800 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:37.800 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:37.801 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:37.801 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-12T19:06:37.801 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_mclock_profile 2026-02-12T19:06:37.861 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:118: TEST_profile_custom_to_builtin: local 'mclock_profile="custom"' 2026-02-12T19:06:37.862 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: eval echo '"custom"' 2026-02-12T19:06:37.862 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: echo custom 2026-02-12T19:06:37.862 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:119: TEST_profile_custom_to_builtin: mclock_profile=custom 2026-02-12T19:06:37.862 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:120: TEST_profile_custom_to_builtin: test custom = custom 2026-02-12T19:06:37.862 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:123: TEST_profile_custom_to_builtin: local client_res 2026-02-12T19:06:37.863 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-12T19:06:37.864 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: bc 2026-02-12T19:06:37.864 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-12T19:06:37.864 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:37.864 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:37.864 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:37.864 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:37.864 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:37.864 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:37.864 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-12T19:06:37.864 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-12T19:06:37.924 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:126: TEST_profile_custom_to_builtin: client_res=0.000000 2026-02-12T19:06:37.925 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:127: TEST_profile_custom_to_builtin: echo 'Original client_res for osd.0 = 0.000000' 2026-02-12T19:06:37.925 INFO:tasks.workunit.client.0.trial010.stdout:Original client_res for osd.0 = 0.000000 2026-02-12T19:06:37.925 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: echo '0.000000 + 0.1' 2026-02-12T19:06:37.926 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: bc -l 2026-02-12T19:06:37.926 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:130: TEST_profile_custom_to_builtin: local client_res_new=.100000 2026-02-12T19:06:37.926 INFO:tasks.workunit.client.0.trial010.stdout:client_res_new = .100000 2026-02-12T19:06:37.927 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:131: TEST_profile_custom_to_builtin: echo 'client_res_new = .100000' 2026-02-12T19:06:37.927 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:132: TEST_profile_custom_to_builtin: ceph config set osd osd_mclock_scheduler_client_res .100000 2026-02-12T19:06:38.150 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-12T19:06:38.375 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:136: TEST_profile_custom_to_builtin: local res=0.100000 2026-02-12T19:06:38.376 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-12T19:06:38.376 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: bc -l 2026-02-12T19:06:38.377 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:137: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-12T19:06:38.378 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-12T19:06:38.378 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: bc 2026-02-12T19:06:38.378 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-12T19:06:38.378 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:38.378 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:38.379 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:38.380 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:38.380 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:38.380 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:38.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-12T19:06:38.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-12T19:06:38.439 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:143: TEST_profile_custom_to_builtin: res=0.100000 2026-02-12T19:06:38.440 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-12T19:06:38.440 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: bc -l 2026-02-12T19:06:38.441 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:144: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-12T19:06:38.441 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:154: TEST_profile_custom_to_builtin: ceph tell osd.0 config set osd_mclock_profile high_recovery_ops 2026-02-12T19:06:38.514 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:06:38.514 INFO:tasks.workunit.client.0.trial010.stdout: "success": "" 2026-02-12T19:06:38.514 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:06:38.522 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: jq .osd_mclock_profile 2026-02-12T19:06:38.523 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-12T19:06:38.523 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:38.524 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:38.524 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:38.524 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:38.524 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:38.524 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:38.524 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-12T19:06:38.524 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_mclock_profile 2026-02-12T19:06:38.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: eval 'mclock_profile="high_recovery_ops"' 2026-02-12T19:06:38.586 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:158: TEST_profile_custom_to_builtin: mclock_profile=high_recovery_ops 2026-02-12T19:06:38.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:160: TEST_profile_custom_to_builtin: test high_recovery_ops = high_recovery_ops 2026-02-12T19:06:38.586 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-12T19:06:38.795 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:165: TEST_profile_custom_to_builtin: local res=0.100000 2026-02-12T19:06:38.796 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-12T19:06:38.796 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: bc -l 2026-02-12T19:06:38.797 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:166: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-12T19:06:38.798 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-12T19:06:38.799 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: bc 2026-02-12T19:06:38.799 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-12T19:06:38.799 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:38.799 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:38.800 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:38.800 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:38.800 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:38.800 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:38.800 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-12T19:06:38.800 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-12T19:06:38.862 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:172: TEST_profile_custom_to_builtin: res=0.100000 2026-02-12T19:06:38.862 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: echo '0.100000 != .100000' 2026-02-12T19:06:38.862 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: bc -l 2026-02-12T19:06:38.863 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:173: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-12T19:06:38.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:178: TEST_profile_custom_to_builtin: ceph config rm osd osd_mclock_scheduler_client_res 2026-02-12T19:06:39.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:180: TEST_profile_custom_to_builtin: sleep 5 2026-02-12T19:06:44.093 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: ceph config get osd.0 osd_mclock_scheduler_client_res 2026-02-12T19:06:44.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:185: TEST_profile_custom_to_builtin: res=0.000000 2026-02-12T19:06:44.309 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: echo '0.000000 != 0.0' 2026-02-12T19:06:44.309 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: bc -l 2026-02-12T19:06:44.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:186: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-12T19:06:44.312 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: jq .osd_mclock_scheduler_client_res 2026-02-12T19:06:44.312 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: bc 2026-02-12T19:06:44.312 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: get_asok_path osd.0 2026-02-12T19:06:44.313 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:44.313 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:44.313 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:44.313 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.313 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.313 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:44.313 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: CEPH_ARGS= 2026-02-12T19:06:44.313 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_mclock_scheduler_client_res 2026-02-12T19:06:44.372 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:193: TEST_profile_custom_to_builtin: res=0.000000 2026-02-12T19:06:44.373 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: echo '0.000000 != 0.000000' 2026-02-12T19:06:44.373 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: bc -l 2026-02-12T19:06:44.374 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:194: TEST_profile_custom_to_builtin: (( 0 )) 2026-02-12T19:06:44.375 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:198: TEST_profile_custom_to_builtin: teardown td/mclock-config 2026-02-12T19:06:44.375 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:44.375 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:44.375 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:44.375 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:44.375 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:44.376 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:44.376 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:44.376 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:44.482 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:44.483 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:44.483 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:44.484 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:44.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:44.485 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:44.485 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:44.486 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:44.486 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:44.487 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:44.487 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:44.487 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:44.489 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:44.489 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:44.493 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:44.493 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.493 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.493 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:44.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:44.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:44.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-12T19:06:44.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:44.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:44.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:44.496 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:44.496 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:44.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:44.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:44.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:44.498 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:44.498 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:44.499 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:44.499 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:44.500 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:44.500 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:44.500 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:44.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:44.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:44.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:44.502 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:44.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:44.504 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:44.504 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:44.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:44.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.506 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:44.507 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:44.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:44.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:31: run: for func in $funcs 2026-02-12T19:06:44.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:32: run: setup td/mclock-config 2026-02-12T19:06:44.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-12T19:06:44.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-12T19:06:44.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:44.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:44.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:44.508 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:44.508 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:44.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:44.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:44.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:44.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:44.510 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:44.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:44.511 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:44.512 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:44.512 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:44.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:44.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:44.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:44.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:44.514 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:44.515 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:44.517 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:44.517 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:44.518 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:44.518 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.518 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:44.519 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:44.520 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:44.520 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-12T19:06:44.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:06:44.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.521 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7506 2026-02-12T19:06:44.523 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:06:44.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:33: run: TEST_recovery_limit_adjustment_mclock td/mclock-config 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:202: TEST_recovery_limit_adjustment_mclock: local dir=td/mclock-config 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:204: TEST_recovery_limit_adjustment_mclock: setup td/mclock-config 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/mclock-config 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/mclock-config 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:44.525 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:44.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:44.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:44.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:44.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:44.527 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:44.527 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:44.528 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:44.528 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:44.529 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:44.529 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:44.530 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:44.530 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:44.531 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:44.531 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:44.532 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:44.533 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:44.533 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:44.534 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:44.534 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.534 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.534 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:44.535 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:44.536 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:44.536 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/mclock-config 2026-02-12T19:06:44.537 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:06:44.537 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.537 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.537 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.7506 2026-02-12T19:06:44.539 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:06:44.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:06:44.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:06:44.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/mclock-config 1' TERM HUP INT 2026-02-12T19:06:44.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:205: TEST_recovery_limit_adjustment_mclock: run_mon td/mclock-config a 2026-02-12T19:06:44.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/mclock-config 2026-02-12T19:06:44.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:06:44.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:06:44.541 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:06:44.541 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/mclock-config/a 2026-02-12T19:06:44.541 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/mclock-config/a --run-dir=td/mclock-config 2026-02-12T19:06:44.560 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:06:44.561 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:44.561 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:44.561 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:44.561 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.561 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.562 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:44.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/mclock-config/a '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --mon-cluster-log-file=td/mclock-config/log --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:06:44.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:06:44.588 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:06:44.588 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:06:44.588 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:06:44.588 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:06:44.589 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:06:44.589 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:06:44.589 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:06:44.589 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:06:44.589 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:44.589 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.590 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.590 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-mon.a.asok 2026-02-12T19:06:44.590 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:06:44.590 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7506/ceph-mon.a.asok config get fsid 2026-02-12T19:06:44.645 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:06:44.645 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:06:44.646 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:06:44.646 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:06:44.646 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:06:44.647 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:06:44.647 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:06:44.647 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:06:44.647 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:44.647 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.647 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.648 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-mon.a.asok 2026-02-12T19:06:44.648 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:06:44.648 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.7506/ceph-mon.a.asok config get mon_host 2026-02-12T19:06:44.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:206: TEST_recovery_limit_adjustment_mclock: run_mgr td/mclock-config x 2026-02-12T19:06:44.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/mclock-config 2026-02-12T19:06:44.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:06:44.703 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:06:44.703 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:06:44.703 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/mclock-config/x 2026-02-12T19:06:44.703 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:06:44.814 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:06:44.814 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:44.814 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:44.814 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:44.814 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.814 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.815 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:44.815 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:06:44.816 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/mclock-config/x '--log-file=td/mclock-config/$name.log' '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --run-dir=td/mclock-config '--pid-file=td/mclock-config/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:06:44.836 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:208: TEST_recovery_limit_adjustment_mclock: run_osd td/mclock-config 0 --osd_op_queue=mclock_scheduler 2026-02-12T19:06:44.836 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/mclock-config 2026-02-12T19:06:44.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:06:44.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:06:44.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:06:44.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/mclock-config/0 2026-02-12T19:06:44.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 ' 2026-02-12T19:06:44.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:06:44.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:06:44.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:06:44.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/mclock-config/0' 2026-02-12T19:06:44.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/mclock-config/0/journal' 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/mclock-config' 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:06:44.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:06:44.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:06:44.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/mclock-config/$name.log' 2026-02-12T19:06:44.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/mclock-config/$name.pid' 2026-02-12T19:06:44.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:06:44.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:06:44.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:06:44.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:06:44.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:06:44.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-12T19:06:44.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/mclock-config/0 2026-02-12T19:06:44.840 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:06:44.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=da371280-a2e1-4738-a35c-620469ffca8a 2026-02-12T19:06:44.840 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 da371280-a2e1-4738-a35c-620469ffca8a 2026-02-12T19:06:44.841 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 da371280-a2e1-4738-a35c-620469ffca8a' 2026-02-12T19:06:44.841 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:06:44.849 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDEJI5ptolOMhAAU+231JlMNupEx9CyNSkQvw== 2026-02-12T19:06:44.849 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDEJI5ptolOMhAAU+231JlMNupEx9CyNSkQvw=="}' 2026-02-12T19:06:44.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new da371280-a2e1-4738-a35c-620469ffca8a -i td/mclock-config/0/new.json 2026-02-12T19:06:44.960 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:44.968 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/mclock-config/0/new.json 2026-02-12T19:06:44.969 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQDEJI5ptolOMhAAU+231JlMNupEx9CyNSkQvw== --osd-uuid da371280-a2e1-4738-a35c-620469ffca8a 2026-02-12T19:06:44.984 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:44.973+0000 7f3be1c04a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:44.985 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:44.977+0000 7f3be1c04a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:44.986 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:44.977+0000 7f3be1c04a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:44.987 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:44.977+0000 7f3be1c04a40 -1 bdev(0x55c5070aa400 td/mclock-config/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:06:44.987 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:44.977+0000 7f3be1c04a40 -1 bluestore(td/mclock-config/0) _read_fsid unparsable uuid 2026-02-12T19:06:45.665 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/mclock-config/0/keyring 2026-02-12T19:06:45.665 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:06:45.666 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:06:45.666 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:06:45.667 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/mclock-config/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:06:45.778 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:06:45.779 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:06:45.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=34c0a0b3-85b9-475c-b201-af87d3a7585b --auth-supported=none --mon-host=127.0.0.1:7124 --debug-mclock 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/mclock-config/0 --osd-journal=td/mclock-config/0/journal --chdir= --run-dir=td/mclock-config '--admin-socket=/tmp/ceph-asok.7506/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/mclock-config/$name.log' '--pid-file=td/mclock-config/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-12T19:06:45.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:06:45.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:06:45.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:06:45.793 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:45.781+0000 7f6fa33fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:45.795 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:45.785+0000 7f6fa33fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:45.796 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:45.785+0000 7f6fa33fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:45.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:06:45.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:06:45.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:06:45.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:06:45.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:06:45.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:45.888 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:45.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:06:45.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:45.889 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:45.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:46.211 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:46.201+0000 7f6fa33fca40 -1 Falling back to public interface 2026-02-12T19:06:46.995 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:06:46.996 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:06:46.996 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:46.996 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:06:46.996 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:46.997 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:47.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:47.275 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:47.265+0000 7f6fa33fca40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:06:48.122 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:06:48.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:48.123 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:06:48.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:06:48.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:48.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:48.341 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4171314132,v1:127.0.0.1:6801/4171314132] [v2:127.0.0.1:6802/4171314132,v1:127.0.0.1:6803/4171314132] exists,up da371280-a2e1-4738-a35c-620469ffca8a 2026-02-12T19:06:48.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:06:48.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:06:48.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:06:48.342 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:48.342 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:48.342 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:48.343 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:48.343 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:48.343 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:48.343 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:48.343 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:48.343 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-12T19:06:48.401 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:210: TEST_recovery_limit_adjustment_mclock: local 'recoveries={"osd_recovery_max_active":"0"}' 2026-02-12T19:06:48.402 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: echo '{"osd_recovery_max_active":"0"}' 2026-02-12T19:06:48.402 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:212: TEST_recovery_limit_adjustment_mclock: grep --quiet osd_recovery_max_active 2026-02-12T19:06:48.403 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:217: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-12T19:06:48.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:218: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-12T19:06:50.629 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:50.629 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:50.629 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:50.629 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:50.629 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:50.629 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:50.630 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:50.630 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:50.630 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-12T19:06:50.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:220: TEST_recovery_limit_adjustment_mclock: local 'max_recoveries={"osd_recovery_max_active":"0"}' 2026-02-12T19:06:50.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:221: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"0"}' = '{"osd_recovery_max_active":"0"}' 2026-02-12T19:06:50.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:225: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_mclock_override_recovery_settings true 2026-02-12T19:06:50.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:226: TEST_recovery_limit_adjustment_mclock: ceph config set osd.0 osd_recovery_max_active 10 2026-02-12T19:06:51.140 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:227: TEST_recovery_limit_adjustment_mclock: sleep 2 2026-02-12T19:06:53.143 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: get_asok_path osd.0 2026-02-12T19:06:53.143 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:06:53.143 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:06:53.144 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:53.144 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:53.145 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:53.145 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.7506/ceph-osd.0.asok 2026-02-12T19:06:53.145 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: CEPH_ARGS= 2026-02-12T19:06:53.145 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: ceph --format=json daemon /tmp/ceph-asok.7506/ceph-osd.0.asok config get osd_recovery_max_active 2026-02-12T19:06:53.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:229: TEST_recovery_limit_adjustment_mclock: max_recoveries='{"osd_recovery_max_active":"10"}' 2026-02-12T19:06:53.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:230: TEST_recovery_limit_adjustment_mclock: test '{"osd_recovery_max_active":"10"}' = '{"osd_recovery_max_active":"10"}' 2026-02-12T19:06:53.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:232: TEST_recovery_limit_adjustment_mclock: teardown td/mclock-config 2026-02-12T19:06:53.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:53.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:53.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:53.205 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:53.206 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:53.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:53.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:53.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:53.313 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:53.313 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:53.314 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:53.314 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:53.315 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:53.316 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:53.316 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:53.317 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:53.317 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:53.318 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:53.318 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:53.318 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:53.319 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:53.320 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:53.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:53.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:53.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:53.323 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:53.325 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:53.325 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:53.325 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/mclock-config.sh:34: run: teardown td/mclock-config 2026-02-12T19:06:53.325 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:53.325 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:53.325 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:53.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:53.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:53.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:53.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:53.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:53.328 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:53.328 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:53.329 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:53.329 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:53.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:53.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:53.331 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:53.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:53.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:53.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:53.333 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:53.334 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:53.335 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:53.335 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:53.336 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:53.336 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:53.336 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:53.336 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:53.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:53.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:53.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-12T19:06:53.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/mclock-config 0 2026-02-12T19:06:53.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/mclock-config 2026-02-12T19:06:53.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-12T19:06:53.339 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/mclock-config KILL 2026-02-12T19:06:53.339 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:53.339 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:53.339 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:53.339 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:53.339 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:53.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:53.341 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:53.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:53.342 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:53.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:53.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:53.344 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:53.345 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:53.345 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:53.345 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:53.345 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:53.346 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:53.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-12T19:06:53.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/mclock-config 2026-02-12T19:06:53.349 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:53.349 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:53.349 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.7506 2026-02-12T19:06:53.349 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.7506 2026-02-12T19:06:53.350 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:53.350 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:53.350 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-12T19:06:53.351 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-12T19:06:53.351 DEBUG:teuthology.orchestra.run.trial010:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-12T19:06:53.402 INFO:tasks.workunit:Running workunit misc/network-ping.sh... 2026-02-12T19:06:53.402 DEBUG:teuthology.orchestra.run.trial010:workunit test misc/network-ping.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh 2026-02-12T19:06:53.459 INFO:tasks.workunit.client.0.trial010.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-12T19:06:53.462 INFO:tasks.workunit.client.0.trial010.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-12T19:06:53.462 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/network-ping 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:6: run: local dir=td/network-ping 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:7: run: shift 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: export CEPH_MON=127.0.0.1:7146 2026-02-12T19:06:53.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:9: run: CEPH_MON=127.0.0.1:7146 2026-02-12T19:06:53.464 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:10: run: export CEPH_ARGS 2026-02-12T19:06:53.464 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: uuidgen 2026-02-12T19:06:53.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:11: run: CEPH_ARGS+='--fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none ' 2026-02-12T19:06:53.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:12: run: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-12T19:06:53.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:13: run: CEPH_ARGS+='--debug_disable_randomized_ping=true ' 2026-02-12T19:06:53.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:14: run: CEPH_ARGS+='--debug_heartbeat_testing_span=5 ' 2026-02-12T19:06:53.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:15: run: CEPH_ARGS+='--osd_heartbeat_interval=1 ' 2026-02-12T19:06:53.465 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: set 2026-02-12T19:06:53.465 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-12T19:06:53.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:16: run: local 'funcs=TEST_network_ping_test1 2026-02-12T19:06:53.467 INFO:tasks.workunit.client.0.trial010.stderr:TEST_network_ping_test2' 2026-02-12T19:06:53.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-12T19:06:53.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-12T19:06:53.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-12T19:06:53.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-12T19:06:53.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-12T19:06:53.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:06:53.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-12T19:06:53.467 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:06:53.468 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:06:53.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:06:53.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:06:53.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:06:53.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:06:53.469 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:06:53.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:06:53.470 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:06:53.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:06:53.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:06:53.472 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:06:53.473 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:53.473 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:06:53.474 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:06:53.474 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:06:53.475 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:06:53.476 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:06:53.476 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-12T19:06:53.477 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:06:53.477 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:53.478 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:06:53.478 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19550 2026-02-12T19:06:53.479 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:06:53.479 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:06:53.479 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-12T19:06:53.480 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:06:53.481 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:53.481 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:06:53.481 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19550 2026-02-12T19:06:53.482 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:06:53.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-12T19:06:53.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-12T19:06:53.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:06:53.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-12T19:06:53.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test1 td/network-ping 2026-02-12T19:06:53.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:25: TEST_network_ping_test1: local dir=td/network-ping 2026-02-12T19:06:53.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:27: TEST_network_ping_test1: run_mon td/network-ping a 2026-02-12T19:06:53.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-12T19:06:53.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:06:53.485 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:06:53.485 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:06:53.485 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-12T19:06:53.485 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-12T19:06:53.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:06:53.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:53.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:53.505 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:53.505 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:53.505 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:06:53.506 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:06:53.506 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:06:53.536 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:06:53.537 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:06:53.538 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:06:53.538 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:06:53.538 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:06:53.539 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:06:53.539 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:06:53.539 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:06:53.539 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:06:53.539 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:53.539 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:53.540 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:06:53.540 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-mon.a.asok 2026-02-12T19:06:53.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:06:53.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19550/ceph-mon.a.asok config get fsid 2026-02-12T19:06:53.595 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:06:53.595 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:06:53.595 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:06:53.595 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:06:53.596 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:06:53.596 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:06:53.596 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:06:53.597 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:06:53.597 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:06:53.597 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:53.597 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:06:53.597 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-mon.a.asok 2026-02-12T19:06:53.597 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:06:53.597 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19550/ceph-mon.a.asok config get mon_host 2026-02-12T19:06:53.652 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:28: TEST_network_ping_test1: run_mgr td/network-ping x 2026-02-12T19:06:53.652 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-12T19:06:53.652 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:06:53.652 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:06:53.652 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:06:53.652 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-12T19:06:53.653 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:06:53.767 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:06:53.767 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:53.767 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:53.767 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:53.767 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:53.767 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:06:53.768 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:06:53.768 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:06:53.768 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:06:53.787 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:29: TEST_network_ping_test1: run_osd td/network-ping 0 2026-02-12T19:06:53.787 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-12T19:06:53.787 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:06:53.787 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:06:53.787 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:06:53.788 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:06:53.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:06:53.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:06:53.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:06:53.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:06:53.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:06:53.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-12T19:06:53.791 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:06:53.791 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=90e4efd9-9ad7-4491-a90b-d184af1291ce 2026-02-12T19:06:53.791 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 90e4efd9-9ad7-4491-a90b-d184af1291ce' 2026-02-12T19:06:53.791 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 90e4efd9-9ad7-4491-a90b-d184af1291ce 2026-02-12T19:06:53.791 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:06:53.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDNJI5p8ZhcLxAALre5lNlbA+ImhEz+I+NIDA== 2026-02-12T19:06:53.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDNJI5p8ZhcLxAALre5lNlbA+ImhEz+I+NIDA=="}' 2026-02-12T19:06:53.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 90e4efd9-9ad7-4491-a90b-d184af1291ce -i td/network-ping/0/new.json 2026-02-12T19:06:53.913 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:53.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-12T19:06:53.922 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDNJI5p8ZhcLxAALre5lNlbA+ImhEz+I+NIDA== --osd-uuid 90e4efd9-9ad7-4491-a90b-d184af1291ce 2026-02-12T19:06:53.939 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:53.929+0000 7fb8f3c22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:53.941 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:53.929+0000 7fb8f3c22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:53.942 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:53.933+0000 7fb8f3c22a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:53.942 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:53.933+0000 7fb8f3c22a40 -1 bdev(0x55e459dec400 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:06:53.942 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:53.933+0000 7fb8f3c22a40 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-12T19:06:54.710 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-12T19:06:54.710 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:06:54.711 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:06:54.712 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:06:54.712 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:06:54.833 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:06:54.834 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:06:54.834 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:06:54.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:06:54.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:06:54.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:06:54.859 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:54.849+0000 7f19c4ab9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:54.861 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:54.849+0000 7f19c4ab9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:54.862 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:54.853+0000 7f19c4ab9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:54.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:06:54.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:06:54.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:06:54.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:06:54.947 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:06:54.947 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:54.947 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:06:54.947 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:54.948 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:54.948 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:55.055 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:55.319 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:55.309+0000 7f19c4ab9a40 -1 Falling back to public interface 2026-02-12T19:06:56.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:06:56.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:56.057 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:06:56.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:06:56.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:56.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:56.252 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:56.241+0000 7f19c4ab9a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:06:56.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:57.274 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:06:57.274 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:57.275 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:06:57.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:06:57.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:57.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:06:57.490 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3845624293,v1:127.0.0.1:6801/3845624293] [v2:127.0.0.1:6802/3845624293,v1:127.0.0.1:6803/3845624293] exists,up 90e4efd9-9ad7-4491-a90b-d184af1291ce 2026-02-12T19:06:57.493 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:06:57.493 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:06:57.493 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:06:57.493 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:30: TEST_network_ping_test1: run_osd td/network-ping 1 2026-02-12T19:06:57.493 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-12T19:06:57.493 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:06:57.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T19:06:57.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:06:57.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-12T19:06:57.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-12T19:06:57.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:06:57.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:06:57.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:06:57.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-12T19:06:57.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-12T19:06:57.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:06:57.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:06:57.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-12T19:06:57.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-12T19:06:57.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:06:57.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:06:57.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:06:57.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:06:57.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:06:57.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:06:57.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-12T19:06:57.496 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:06:57.497 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=17bfeb38-2606-41db-855b-93838fbbd026 2026-02-12T19:06:57.497 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 17bfeb38-2606-41db-855b-93838fbbd026' 2026-02-12T19:06:57.497 INFO:tasks.workunit.client.0.trial010.stdout:add osd1 17bfeb38-2606-41db-855b-93838fbbd026 2026-02-12T19:06:57.498 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:06:57.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDRJI5pccbqHRAA+1n6JvVGXJc6qipO0VUcpw== 2026-02-12T19:06:57.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDRJI5pccbqHRAA+1n6JvVGXJc6qipO0VUcpw=="}' 2026-02-12T19:06:57.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 17bfeb38-2606-41db-855b-93838fbbd026 -i td/network-ping/1/new.json 2026-02-12T19:06:57.733 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:06:57.742 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-12T19:06:57.743 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDRJI5pccbqHRAA+1n6JvVGXJc6qipO0VUcpw== --osd-uuid 17bfeb38-2606-41db-855b-93838fbbd026 2026-02-12T19:06:57.760 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:57.749+0000 7fb084a75a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:57.762 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:57.753+0000 7fb084a75a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:57.763 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:57.753+0000 7fb084a75a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:57.764 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:57.753+0000 7fb084a75a40 -1 bdev(0x55a9b01d4400 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-12T19:06:57.764 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:57.753+0000 7fb084a75a40 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-12T19:06:58.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-12T19:06:58.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:06:58.398 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T19:06:58.398 INFO:tasks.workunit.client.0.trial010.stdout:adding osd1 key to auth repository 2026-02-12T19:06:58.398 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:06:58.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T19:06:58.688 INFO:tasks.workunit.client.0.trial010.stdout:start osd.1 2026-02-12T19:06:58.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:06:58.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:06:58.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:06:58.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:06:58.712 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:58.701+0000 7f9177bcaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:58.714 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:58.705+0000 7f9177bcaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:58.716 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:58.705+0000 7f9177bcaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:06:58.897 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T19:06:58.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:06:58.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T19:06:58.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:06:58.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:06:58.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:06:58.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:06:58.898 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:06:58.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:06:58.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:06:59.095 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:59.085+0000 7f9177bcaa40 -1 Falling back to public interface 2026-02-12T19:06:59.114 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:06:59.683 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:06:59.673+0000 7f9177bcaa40 -1 osd.1 0 log_to_monitors true 2026-02-12T19:07:00.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:07:00.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:07:00.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:07:00.116 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:07:00.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:07:00.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:07:00.336 INFO:tasks.workunit.client.0.trial010.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2681043312,v1:127.0.0.1:6811/2681043312] [v2:127.0.0.1:6812/2681043312,v1:127.0.0.1:6813/2681043312] exists,up 17bfeb38-2606-41db-855b-93838fbbd026 2026-02-12T19:07:00.339 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:07:00.339 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:07:00.339 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:07:00.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:31: TEST_network_ping_test1: run_osd td/network-ping 2 2026-02-12T19:07:00.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-12T19:07:00.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:07:00.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T19:07:00.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:07:00.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-12T19:07:00.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-12T19:07:00.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:07:00.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:07:00.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:07:00.341 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:07:00.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-12T19:07:00.343 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:07:00.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=eb8cdf59-f604-4e06-8c8d-096fdc72c24a 2026-02-12T19:07:00.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 eb8cdf59-f604-4e06-8c8d-096fdc72c24a' 2026-02-12T19:07:00.344 INFO:tasks.workunit.client.0.trial010.stdout:add osd2 eb8cdf59-f604-4e06-8c8d-096fdc72c24a 2026-02-12T19:07:00.344 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:07:00.353 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDUJI5pdXi6FBAACxns9lsdBvdRUn9oalW73A== 2026-02-12T19:07:00.353 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDUJI5pdXi6FBAACxns9lsdBvdRUn9oalW73A=="}' 2026-02-12T19:07:00.353 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new eb8cdf59-f604-4e06-8c8d-096fdc72c24a -i td/network-ping/2/new.json 2026-02-12T19:07:00.578 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:07:00.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-12T19:07:00.588 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDUJI5pdXi6FBAACxns9lsdBvdRUn9oalW73A== --osd-uuid eb8cdf59-f604-4e06-8c8d-096fdc72c24a 2026-02-12T19:07:00.605 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:07:00.593+0000 7fd572718a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:07:00.606 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:07:00.597+0000 7fd572718a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:07:00.607 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:07:00.597+0000 7fd572718a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:07:00.608 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:07:00.597+0000 7fd572718a40 -1 bdev(0x55dbe86d4400 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-12T19:07:00.608 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:07:00.597+0000 7fd572718a40 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-12T19:07:01.273 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-12T19:07:01.273 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:07:01.274 INFO:tasks.workunit.client.0.trial010.stdout:adding osd2 key to auth repository 2026-02-12T19:07:01.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T19:07:01.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:07:01.579 INFO:tasks.workunit.client.0.trial010.stdout:start osd.2 2026-02-12T19:07:01.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T19:07:01.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:07:01.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:07:01.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:07:01.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:07:01.603 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:07:01.597+0000 7f6978517a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:07:01.604 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:07:01.597+0000 7f6978517a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:07:01.606 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:07:01.597+0000 7f6978517a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:07:01.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T19:07:01.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:07:01.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T19:07:01.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:07:01.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:07:01.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:07:01.789 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:07:01.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:07:01.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:07:01.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T19:07:01.955 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:07:01.949+0000 7f6978517a40 -1 Falling back to public interface 2026-02-12T19:07:02.007 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:07:02.556 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:07:02.549+0000 7f6978517a40 -1 osd.2 0 log_to_monitors true 2026-02-12T19:07:03.009 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:07:03.009 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:07:03.009 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:07:03.010 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:07:03.010 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:07:03.010 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T19:07:03.226 INFO:tasks.workunit.client.0.trial010.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2341882022,v1:127.0.0.1:6819/2341882022] [v2:127.0.0.1:6820/2341882022,v1:127.0.0.1:6821/2341882022] exists,up eb8cdf59-f604-4e06-8c8d-096fdc72c24a 2026-02-12T19:07:03.227 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:07:03.227 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:07:03.227 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:07:03.227 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:33: TEST_network_ping_test1: sleep 5 2026-02-12T19:07:08.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:35: TEST_network_ping_test1: create_pool foo 16 2026-02-12T19:07:08.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-12T19:07:08.495 INFO:tasks.workunit.client.0.trial010.stderr:pool 'foo' already exists 2026-02-12T19:07:08.505 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:07:09.506 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:38: TEST_network_ping_test1: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-12T19:07:09.527 INFO:tasks.workunit.client.0.trial010.stdout:hints = 1 2026-02-12T19:07:09.527 INFO:tasks.workunit.client.0.trial010.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-12T19:07:09.527 INFO:tasks.workunit.client.0.trial010.stdout:Object prefix: benchmark_data_trial010_25301 2026-02-12T19:07:09.527 INFO:tasks.workunit.client.0.trial010.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T19:07:09.527 INFO:tasks.workunit.client.0.trial010.stdout: 0 0 0 0 0 0 - 0 2026-02-12T19:07:10.527 INFO:tasks.workunit.client.0.trial010.stdout: 1 16 10323 10307 40.2581 40.2617 0.00109404 0.00155011 2026-02-12T19:07:11.528 INFO:tasks.workunit.client.0.trial010.stdout: 2 16 21200 21184 41.3715 42.4883 0.00132274 0.00150886 2026-02-12T19:07:12.528 INFO:tasks.workunit.client.0.trial010.stdout: 3 15 32549 32534 42.3585 44.3359 0.000979627 0.00147376 2026-02-12T19:07:13.528 INFO:tasks.workunit.client.0.trial010.stdout: 4 16 43437 43421 42.3999 42.5273 0.00281324 0.00147231 2026-02-12T19:07:14.528 INFO:tasks.workunit.client.0.trial010.stdout: 5 16 54613 54597 42.6505 43.6562 0.00192569 0.00146373 2026-02-12T19:07:15.528 INFO:tasks.workunit.client.0.trial010.stdout: 6 16 65367 65351 42.5428 42.0078 0.00103553 0.00146759 2026-02-12T19:07:16.528 INFO:tasks.workunit.client.0.trial010.stdout: 7 16 76219 76203 42.5206 42.3906 0.00133163 0.00146838 2026-02-12T19:07:17.528 INFO:tasks.workunit.client.0.trial010.stdout: 8 16 86667 86651 42.3067 40.8125 0.00135342 0.00147573 2026-02-12T19:07:18.528 INFO:tasks.workunit.client.0.trial010.stdout: 9 15 96261 96246 41.7702 37.4805 0.00179232 0.00149475 2026-02-12T19:07:19.528 INFO:tasks.workunit.client.0.trial010.stdout:Total time run: 10.0008 2026-02-12T19:07:19.528 INFO:tasks.workunit.client.0.trial010.stdout:Total writes made: 105368 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Write size: 4096 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Object size: 4096 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Bandwidth (MB/sec): 41.1563 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Stddev Bandwidth: 2.03914 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Max bandwidth (MB/sec): 44.3359 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Min bandwidth (MB/sec): 37.4805 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Average IOPS: 10536 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Stddev IOPS: 522.021 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Max IOPS: 11350 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Min IOPS: 9595 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Average Latency(s): 0.00151715 2026-02-12T19:07:19.529 INFO:tasks.workunit.client.0.trial010.stdout:Stddev Latency(s): 0.000739422 2026-02-12T19:07:19.530 INFO:tasks.workunit.client.0.trial010.stdout:Max latency(s): 0.0155179 2026-02-12T19:07:19.530 INFO:tasks.workunit.client.0.trial010.stdout:Min latency(s): 0.000440242 2026-02-12T19:07:19.531 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:41: TEST_network_ping_test1: sleep 10 2026-02-12T19:07:29.533 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:42: TEST_network_ping_test1: flush_pg_stats 2026-02-12T19:07:29.533 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:07:29.534 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:07:29.744 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T19:07:29.744 INFO:tasks.workunit.client.0.trial010.stderr:1 2026-02-12T19:07:29.744 INFO:tasks.workunit.client.0.trial010.stderr:2' 2026-02-12T19:07:29.744 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:07:29.745 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:07:29.745 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:07:29.826 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-12T19:07:29.826 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-12T19:07:29.826 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836489' 2026-02-12T19:07:29.826 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:07:29.827 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T19:07:29.903 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705672 2026-02-12T19:07:29.903 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705672 2026-02-12T19:07:29.904 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836489 1-38654705672' 2026-02-12T19:07:29.904 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:07:29.904 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T19:07:29.983 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574856 2026-02-12T19:07:29.983 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574856 2026-02-12T19:07:29.983 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-55834574856' 2026-02-12T19:07:29.983 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:07:29.984 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836489 2026-02-12T19:07:29.984 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:07:29.984 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:07:29.985 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-12T19:07:29.986 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:07:29.987 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836489 2026-02-12T19:07:29.987 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-12T19:07:29.987 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836489 2026-02-12T19:07:29.988 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:07:30.202 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-12T19:07:30.202 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:07:30.203 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705672 2026-02-12T19:07:30.203 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:07:30.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T19:07:30.205 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705672 2026-02-12T19:07:30.205 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:07:30.207 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705672 2026-02-12T19:07:30.207 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.1 seq 38654705672 2026-02-12T19:07:30.207 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-12T19:07:30.208 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T19:07:30.422 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-12T19:07:30.422 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:07:30.422 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574856 2026-02-12T19:07:30.423 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:07:30.423 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T19:07:30.424 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574856 2026-02-12T19:07:30.424 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:07:30.426 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574856 2026-02-12T19:07:30.426 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574856' 2026-02-12T19:07:30.426 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.2 seq 55834574856 2026-02-12T19:07:30.426 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T19:07:30.647 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574856 -lt 55834574856 2026-02-12T19:07:30.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:07:30.649 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: get_asok_path osd.0 2026-02-12T19:07:30.649 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:07:30.649 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:07:30.649 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:07:30.649 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:07:30.649 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:07:30.649 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-osd.0.asok 2026-02-12T19:07:30.650 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:07:30.650 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:44: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-osd.0.asok dump_osd_network 2026-02-12T19:07:30.704 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:07:30.704 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 1000, 2026-02-12T19:07:30.704 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [] 2026-02-12T19:07:30.705 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:07:30.711 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:30.711 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:07:30.721 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:45: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:07:30.722 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:30.723 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: jq .threshold 2026-02-12T19:07:30.731 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:46: TEST_network_ping_test1: test 1000 = 1000 2026-02-12T19:07:30.732 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:07:30.733 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-12T19:07:30.733 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-12T19:07:30.733 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-12T19:07:30.734 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:07:30.734 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:07:30.734 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:07:30.734 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-mgr.x.asok 2026-02-12T19:07:30.734 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:07:30.734 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:48: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-mgr.x.asok dump_osd_network 2026-02-12T19:07:30.782 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:07:30.782 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 1000, 2026-02-12T19:07:30.782 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [] 2026-02-12T19:07:30.782 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:07:30.788 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:30.789 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:07:30.798 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:49: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:07:30.799 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:30.799 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: jq .threshold 2026-02-12T19:07:30.810 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:50: TEST_network_ping_test1: test 1000 = 1000 2026-02-12T19:07:30.810 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:07:30.811 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: get_asok_path osd.0 2026-02-12T19:07:30.811 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:07:30.811 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:07:30.812 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:07:30.812 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:07:30.812 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:07:30.812 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-osd.0.asok 2026-02-12T19:07:30.812 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:07:30.813 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:52: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-osd.0.asok dump_osd_network 0 2026-02-12T19:07:30.864 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:07:30.864 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 0, 2026-02-12T19:07:30.864 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [ 2026-02-12T19:07:30.864 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.864 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:27 2026", 2026-02-12T19:07:30.864 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.155, 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.235, 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.219 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.179, 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 1.247, 2026-02-12T19:07:30.865 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.247 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.179, 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 1.247, 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.247 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.222 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:28 2026", 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.866 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.197, 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.196, 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.221 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.232, 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.494, 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.494 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.867 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.232, 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.494, 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.494 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.256 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:28 2026", 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.868 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.173, 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.191, 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.216 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.205, 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.457, 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.457 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.205, 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.457, 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.457 2026-02-12T19:07:30.869 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.238 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:27 2026", 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.132, 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.206, 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.175 2026-02-12T19:07:30.870 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.141, 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 1.222, 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.222 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.141, 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 1.222, 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.222 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.206 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: } 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout: ] 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:30.871 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:07:30.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:53: TEST_network_ping_test1: test 4 = 4 2026-02-12T19:07:30.882 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:30.883 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: jq .threshold 2026-02-12T19:07:30.891 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:54: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:07:30.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:07:30.893 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-12T19:07:30.893 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-12T19:07:30.893 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-12T19:07:30.894 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:07:30.894 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:07:30.894 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:07:30.894 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-mgr.x.asok 2026-02-12T19:07:30.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:07:30.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:56: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-mgr.x.asok dump_osd_network 0 2026-02-12T19:07:30.940 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:07:30.940 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 0, 2026-02-12T19:07:30.940 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [ 2026-02-12T19:07:30.940 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.940 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:25 2026", 2026-02-12T19:07:30.940 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.940 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.123, 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.357, 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.267 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.106, 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.100, 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.100 2026-02-12T19:07:30.941 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.151, 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 5.098, 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 5.098 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.140 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:25 2026", 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:07:30.942 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.106, 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.335, 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.244 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.085, 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.085, 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.085 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.943 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.141, 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 5.072, 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 5.072 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.109 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:29 2026", 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.169, 2026-02-12T19:07:30.944 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.261, 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.203 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.159, 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.115, 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.115 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.177, 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 1.934, 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.934 2026-02-12T19:07:30.945 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.172 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:27 2026", 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.155, 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.235, 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.219 2026-02-12T19:07:30.946 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.947 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.947 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.121, 2026-02-12T19:07:30.947 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.121, 2026-02-12T19:07:30.947 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.121 2026-02-12T19:07:30.947 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.947 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.947 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.179, 2026-02-12T19:07:30.947 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 1.247, 2026-02-12T19:07:30.947 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.247 2026-02-12T19:07:30.948 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.948 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.165 2026-02-12T19:07:30.948 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.948 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.948 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:28 2026", 2026-02-12T19:07:30.948 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.948 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:07:30.948 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:07:30.948 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:30.948 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.949 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.148, 2026-02-12T19:07:30.949 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.234, 2026-02-12T19:07:30.949 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.167 2026-02-12T19:07:30.949 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.949 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.949 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.119, 2026-02-12T19:07:30.949 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.113, 2026-02-12T19:07:30.949 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.113 2026-02-12T19:07:30.949 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.949 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.949 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.199, 2026-02-12T19:07:30.950 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 2.026, 2026-02-12T19:07:30.950 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 2.026 2026-02-12T19:07:30.950 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.950 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.135 2026-02-12T19:07:30.950 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.950 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.950 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:28 2026", 2026-02-12T19:07:30.950 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.950 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:30.950 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:30.951 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:30.951 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.951 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.197, 2026-02-12T19:07:30.951 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.196, 2026-02-12T19:07:30.951 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.221 2026-02-12T19:07:30.951 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.951 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.951 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.167, 2026-02-12T19:07:30.951 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.109, 2026-02-12T19:07:30.951 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.109 2026-02-12T19:07:30.951 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.952 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.952 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.232, 2026-02-12T19:07:30.952 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.494, 2026-02-12T19:07:30.952 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.494 2026-02-12T19:07:30.952 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.952 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.178 2026-02-12T19:07:30.952 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.952 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.952 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:28 2026", 2026-02-12T19:07:30.952 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.952 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:30.953 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:30.953 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:30.953 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.953 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.173, 2026-02-12T19:07:30.953 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.191, 2026-02-12T19:07:30.953 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.216 2026-02-12T19:07:30.953 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.953 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.953 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.130, 2026-02-12T19:07:30.953 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.126, 2026-02-12T19:07:30.953 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.126 2026-02-12T19:07:30.954 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.954 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.954 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.205, 2026-02-12T19:07:30.954 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.457, 2026-02-12T19:07:30.954 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.457 2026-02-12T19:07:30.954 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.954 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.133 2026-02-12T19:07:30.954 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.954 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.954 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:28 2026", 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.141, 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.206, 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.150 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.092, 2026-02-12T19:07:30.955 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.092, 2026-02-12T19:07:30.956 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.092 2026-02-12T19:07:30.956 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.956 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.956 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.189, 2026-02-12T19:07:30.956 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 1.993, 2026-02-12T19:07:30.956 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.993 2026-02-12T19:07:30.956 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.956 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.153 2026-02-12T19:07:30.956 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.956 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.956 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:27 2026", 2026-02-12T19:07:30.957 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.957 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:30.957 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:30.957 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:30.957 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.957 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.132, 2026-02-12T19:07:30.957 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.206, 2026-02-12T19:07:30.957 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.175 2026-02-12T19:07:30.957 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.957 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.957 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.120, 2026-02-12T19:07:30.958 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.118, 2026-02-12T19:07:30.958 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.118 2026-02-12T19:07:30.958 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.958 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.958 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.141, 2026-02-12T19:07:30.958 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 1.222, 2026-02-12T19:07:30.958 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.222 2026-02-12T19:07:30.958 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.958 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.146 2026-02-12T19:07:30.958 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.958 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.959 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:28 2026", 2026-02-12T19:07:30.959 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.959 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:07:30.959 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:30.959 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:30.959 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.959 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.167, 2026-02-12T19:07:30.959 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.181, 2026-02-12T19:07:30.959 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.183 2026-02-12T19:07:30.959 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.959 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.960 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.153, 2026-02-12T19:07:30.960 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.134, 2026-02-12T19:07:30.960 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.134 2026-02-12T19:07:30.960 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.960 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.960 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.202, 2026-02-12T19:07:30.960 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.341, 2026-02-12T19:07:30.960 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.341 2026-02-12T19:07:30.960 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.960 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.184 2026-02-12T19:07:30.960 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.961 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.961 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:29 2026", 2026-02-12T19:07:30.961 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.961 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:07:30.961 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:30.961 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:30.961 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.961 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.119, 2026-02-12T19:07:30.961 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.152, 2026-02-12T19:07:30.961 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.178 2026-02-12T19:07:30.962 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.962 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.962 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.099, 2026-02-12T19:07:30.962 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.099, 2026-02-12T19:07:30.962 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.099 2026-02-12T19:07:30.962 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.962 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.962 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.130, 2026-02-12T19:07:30.962 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.322, 2026-02-12T19:07:30.962 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.322 2026-02-12T19:07:30.962 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.128 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:28 2026", 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.133, 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.162, 2026-02-12T19:07:30.963 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.160 2026-02-12T19:07:30.964 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.964 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:30.964 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.111, 2026-02-12T19:07:30.964 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.111, 2026-02-12T19:07:30.964 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.111 2026-02-12T19:07:30.964 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.964 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:30.964 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.142, 2026-02-12T19:07:30.964 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.249, 2026-02-12T19:07:30.964 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.249 2026-02-12T19:07:30.964 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:30.965 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.174 2026-02-12T19:07:30.965 INFO:tasks.workunit.client.0.trial010.stdout: } 2026-02-12T19:07:30.965 INFO:tasks.workunit.client.0.trial010.stdout: ] 2026-02-12T19:07:30.965 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:07:30.966 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:30.966 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:07:30.966 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:57: TEST_network_ping_test1: test 12 = 12 2026-02-12T19:07:30.966 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:30.967 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: jq .threshold 2026-02-12T19:07:30.969 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:58: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:07:30.969 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:61: TEST_network_ping_test1: sleep 20 2026-02-12T19:07:50.971 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:62: TEST_network_ping_test1: flush_pg_stats 2026-02-12T19:07:50.971 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:07:50.972 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:07:51.188 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T19:07:51.188 INFO:tasks.workunit.client.0.trial010.stderr:1 2026-02-12T19:07:51.188 INFO:tasks.workunit.client.0.trial010.stderr:2' 2026-02-12T19:07:51.188 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:07:51.189 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:07:51.189 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:07:51.268 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836495 2026-02-12T19:07:51.268 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836495 2026-02-12T19:07:51.268 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495' 2026-02-12T19:07:51.268 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:07:51.269 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T19:07:51.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705679 2026-02-12T19:07:51.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705679 2026-02-12T19:07:51.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679' 2026-02-12T19:07:51.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:07:51.348 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T19:07:51.427 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574862 2026-02-12T19:07:51.428 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574862 2026-02-12T19:07:51.428 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862' 2026-02-12T19:07:51.428 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:07:51.428 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836495 2026-02-12T19:07:51.428 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:07:51.429 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:07:51.430 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836495 2026-02-12T19:07:51.430 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:07:51.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836495 2026-02-12T19:07:51.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2026-02-12T19:07:51.432 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836495 2026-02-12T19:07:51.432 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:07:51.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836493 -lt 21474836495 2026-02-12T19:07:51.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:07:52.647 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:07:52.648 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:07:52.859 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836495 -lt 21474836495 2026-02-12T19:07:52.860 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:07:52.860 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705679 2026-02-12T19:07:52.860 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:07:52.862 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T19:07:52.863 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705679 2026-02-12T19:07:52.863 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:07:52.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705679 2026-02-12T19:07:52.864 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.1 seq 38654705679 2026-02-12T19:07:52.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705679' 2026-02-12T19:07:52.865 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T19:07:53.087 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705679 -lt 38654705679 2026-02-12T19:07:53.087 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:07:53.087 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574862 2026-02-12T19:07:53.088 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:07:53.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T19:07:53.089 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574862 2026-02-12T19:07:53.089 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:07:53.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574862 2026-02-12T19:07:53.091 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.2 seq 55834574862 2026-02-12T19:07:53.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574862' 2026-02-12T19:07:53.092 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T19:07:53.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574862 -lt 55834574862 2026-02-12T19:07:53.311 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:07:53.311 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: get_asok_path osd.0 2026-02-12T19:07:53.311 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:07:53.311 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:07:53.312 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:07:53.312 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:07:53.312 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:07:53.312 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-osd.0.asok 2026-02-12T19:07:53.312 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:07:53.312 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:63: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-osd.0.asok dump_osd_network 2026-02-12T19:07:53.367 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:07:53.367 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 1000, 2026-02-12T19:07:53.367 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [] 2026-02-12T19:07:53.367 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:07:53.373 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:53.374 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:07:53.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:64: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:07:53.383 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:53.384 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: jq .threshold 2026-02-12T19:07:53.394 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:65: TEST_network_ping_test1: test 1000 = 1000 2026-02-12T19:07:53.395 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:07:53.395 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-12T19:07:53.395 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-12T19:07:53.395 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-12T19:07:53.396 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:07:53.396 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:07:53.396 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:07:53.396 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-mgr.x.asok 2026-02-12T19:07:53.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:07:53.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:67: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-mgr.x.asok dump_osd_network 2026-02-12T19:07:53.443 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:07:53.443 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 1000, 2026-02-12T19:07:53.443 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [] 2026-02-12T19:07:53.443 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:07:53.450 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:53.450 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:07:53.459 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:68: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:07:53.460 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:53.460 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: jq .threshold 2026-02-12T19:07:53.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:69: TEST_network_ping_test1: test 1000 = 1000 2026-02-12T19:07:53.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:07:53.471 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: get_asok_path osd.0 2026-02-12T19:07:53.471 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:07:53.471 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:07:53.471 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:07:53.471 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:07:53.471 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:07:53.471 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-osd.0.asok 2026-02-12T19:07:53.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:07:53.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:71: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-osd.0.asok dump_osd_network 0 2026-02-12T19:07:53.523 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 0, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [ 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:53 2026", 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.202, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.187, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.206 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.228, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.333, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.494 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.228, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.333, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.494 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.228 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:53 2026", 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.206, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.183, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.201 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.240, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.320, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.457 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.524 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.240, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.320, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.457 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.240 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:52 2026", 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.161, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.159, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.202 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.184, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.281, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.247 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.184, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.281, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.247 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.215 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:52 2026", 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.142, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.141, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.169 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.195, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.297, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.222 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.525 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.195, 2026-02-12T19:07:53.526 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.297, 2026-02-12T19:07:53.526 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.222 2026-02-12T19:07:53.526 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.526 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.256 2026-02-12T19:07:53.526 INFO:tasks.workunit.client.0.trial010.stdout: } 2026-02-12T19:07:53.526 INFO:tasks.workunit.client.0.trial010.stdout: ] 2026-02-12T19:07:53.526 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:07:53.530 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:53.530 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:07:53.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:72: TEST_network_ping_test1: test 4 = 4 2026-02-12T19:07:53.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:53.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: jq .threshold 2026-02-12T19:07:53.550 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:73: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:07:53.550 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:07:53.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-12T19:07:53.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-12T19:07:53.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-12T19:07:53.551 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:07:53.551 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:07:53.551 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:07:53.552 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-mgr.x.asok 2026-02-12T19:07:53.552 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:07:53.552 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:75: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-mgr.x.asok dump_osd_network 0 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 0, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [ 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:50 2026", 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.113, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.124, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.234 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.104, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.104, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.100 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.127, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.173, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 5.098 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.114 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:50 2026", 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.124, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.116, 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.216 2026-02-12T19:07:53.599 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.098, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.085, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.085 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.143, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.149, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 5.072 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.127 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:48 2026", 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.176, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.186, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.208 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.166, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.148, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.109 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.183, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.333, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.494 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.197 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:47 2026", 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.144, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.158, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.205 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.600 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.132, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.121, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.121 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.165, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.281, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.247 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.168 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:48 2026", 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.171, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.177, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.202 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.160, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.130, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.126 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.177, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.320, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.457 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.185 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:49 2026", 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.150, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.140, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.192 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.601 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.131, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.096, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.096 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.200, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.209, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.934 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.108 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:48 2026", 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.148, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.153, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.174 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.117, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.113, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.113 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.176, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.227, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.341 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.162 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:47 2026", 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.131, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.139, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.170 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.119, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.109, 2026-02-12T19:07:53.602 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.109 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.149, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.297, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.222 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.133 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:48 2026", 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.125, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.132, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.165 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.099, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.099, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.099 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.147, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.210, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 2.026 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.178 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:49 2026", 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.134, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.131, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.163 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.120, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.099, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.099 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.603 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.152, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.179, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.322 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.158 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:48 2026", 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.141, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.147, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.158 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.109, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.109, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.109 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.165, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.198, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.249 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.196 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:07:48 2026", 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.102, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.114, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.146 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.087, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.086, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.086 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.119, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.189, 2026-02-12T19:07:53.604 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 1.993 2026-02-12T19:07:53.605 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:07:53.605 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.135 2026-02-12T19:07:53.605 INFO:tasks.workunit.client.0.trial010.stdout: } 2026-02-12T19:07:53.605 INFO:tasks.workunit.client.0.trial010.stdout: ] 2026-02-12T19:07:53.605 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:07:53.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:53.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:07:53.614 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:76: TEST_network_ping_test1: test 12 = 12 2026-02-12T19:07:53.615 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:07:53.616 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: jq .threshold 2026-02-12T19:07:53.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:77: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:07:53.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:81: TEST_network_ping_test1: sleep 50 2026-02-12T19:08:43.624 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:82: TEST_network_ping_test1: flush_pg_stats 2026-02-12T19:08:43.625 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:08:43.625 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:08:43.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T19:08:43.830 INFO:tasks.workunit.client.0.trial010.stderr:1 2026-02-12T19:08:43.831 INFO:tasks.workunit.client.0.trial010.stderr:2' 2026-02-12T19:08:43.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:08:43.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:08:43.831 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:08:43.913 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836508 2026-02-12T19:08:43.914 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836508 2026-02-12T19:08:43.914 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508' 2026-02-12T19:08:43.914 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:08:43.914 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T19:08:43.996 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705691 2026-02-12T19:08:43.996 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705691 2026-02-12T19:08:43.996 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691' 2026-02-12T19:08:43.996 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:08:43.996 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T19:08:44.073 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574875 2026-02-12T19:08:44.073 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574875 2026-02-12T19:08:44.073 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836508 1-38654705691 2-55834574875' 2026-02-12T19:08:44.073 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:08:44.074 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836508 2026-02-12T19:08:44.074 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:08:44.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:08:44.076 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836508 2026-02-12T19:08:44.076 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:08:44.077 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836508 2026-02-12T19:08:44.077 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836508 2026-02-12T19:08:44.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836508' 2026-02-12T19:08:44.078 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:08:44.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836508 -lt 21474836508 2026-02-12T19:08:44.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:08:44.293 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705691 2026-02-12T19:08:44.293 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:08:44.294 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T19:08:44.295 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705691 2026-02-12T19:08:44.295 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:08:44.296 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705691 2026-02-12T19:08:44.297 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.1 seq 38654705691 2026-02-12T19:08:44.297 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705691' 2026-02-12T19:08:44.297 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T19:08:44.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705691 -lt 38654705691 2026-02-12T19:08:44.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:08:44.510 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-55834574875 2026-02-12T19:08:44.510 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:08:44.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T19:08:44.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574875 2026-02-12T19:08:44.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:08:44.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=55834574875 2026-02-12T19:08:44.513 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.2 seq 55834574875 2026-02-12T19:08:44.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 55834574875' 2026-02-12T19:08:44.514 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T19:08:44.740 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574873 -lt 55834574875 2026-02-12T19:08:44.740 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:08:45.741 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:08:45.742 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T19:08:45.952 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574873 -lt 55834574875 2026-02-12T19:08:45.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:08:46.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T19:08:46.954 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T19:08:47.164 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 55834574875 -lt 55834574875 2026-02-12T19:08:47.166 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:08:47.166 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: get_asok_path osd.0 2026-02-12T19:08:47.166 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:08:47.166 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:08:47.167 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:08:47.167 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:47.167 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:47.167 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-osd.0.asok 2026-02-12T19:08:47.167 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:08:47.168 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:83: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-osd.0.asok dump_osd_network 2026-02-12T19:08:47.221 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:08:47.221 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 1000, 2026-02-12T19:08:47.221 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [] 2026-02-12T19:08:47.221 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:08:47.228 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:08:47.228 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:08:47.236 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:84: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:08:47.237 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:08:47.237 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: jq .threshold 2026-02-12T19:08:47.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:85: TEST_network_ping_test1: test 1000 = 1000 2026-02-12T19:08:47.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:08:47.249 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-12T19:08:47.249 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-12T19:08:47.249 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-12T19:08:47.250 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:08:47.250 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:47.251 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:47.251 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-mgr.x.asok 2026-02-12T19:08:47.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:08:47.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:87: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-mgr.x.asok dump_osd_network 2026-02-12T19:08:47.297 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:08:47.297 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 1000, 2026-02-12T19:08:47.297 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [] 2026-02-12T19:08:47.297 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:08:47.304 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:08:47.304 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:08:47.314 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:88: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:08:47.315 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:08:47.315 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: jq .threshold 2026-02-12T19:08:47.324 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:89: TEST_network_ping_test1: test 1000 = 1000 2026-02-12T19:08:47.324 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:08:47.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: get_asok_path osd.0 2026-02-12T19:08:47.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:08:47.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:08:47.326 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:08:47.326 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:47.326 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:47.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-osd.0.asok 2026-02-12T19:08:47.327 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:08:47.327 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:91: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-osd.0.asok dump_osd_network 0 2026-02-12T19:08:47.378 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:08:47.379 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 0, 2026-02-12T19:08:47.379 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [ 2026-02-12T19:08:47.379 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.379 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:47 2026", 2026-02-12T19:08:47.379 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.379 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:08:47.379 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:08:47.379 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:08:47.379 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.379 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.207, 2026-02-12T19:08:47.379 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.148, 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.140 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.248, 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.248, 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.296 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.248, 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.248, 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.296 2026-02-12T19:08:47.380 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.216 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:47 2026", 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.197, 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.156, 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.153 2026-02-12T19:08:47.381 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.270, 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.270, 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.327 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.270, 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.270, 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.327 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.173 2026-02-12T19:08:47.382 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:43 2026", 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.143, 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.172, 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.176 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.383 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.187, 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.217, 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.333 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.187, 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.217, 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.333 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.185 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.384 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:43 2026", 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.157, 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.162, 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.169 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.172, 2026-02-12T19:08:47.385 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.202, 2026-02-12T19:08:47.386 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.320 2026-02-12T19:08:47.386 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.386 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.386 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.172, 2026-02-12T19:08:47.386 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.202, 2026-02-12T19:08:47.386 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.320 2026-02-12T19:08:47.386 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.386 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.199 2026-02-12T19:08:47.386 INFO:tasks.workunit.client.0.trial010.stdout: } 2026-02-12T19:08:47.386 INFO:tasks.workunit.client.0.trial010.stdout: ] 2026-02-12T19:08:47.386 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:08:47.387 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:08:47.387 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:08:47.395 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:92: TEST_network_ping_test1: test 4 = 4 2026-02-12T19:08:47.396 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:08:47.396 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: jq .threshold 2026-02-12T19:08:47.406 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:93: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:08:47.407 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:08:47.407 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-12T19:08:47.408 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-12T19:08:47.408 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-12T19:08:47.409 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:08:47.409 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:47.409 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:47.409 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-mgr.x.asok 2026-02-12T19:08:47.409 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:08:47.409 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:95: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-mgr.x.asok dump_osd_network 0 2026-02-12T19:08:47.456 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:08:47.456 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 0, 2026-02-12T19:08:47.456 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [ 2026-02-12T19:08:47.456 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.456 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:43 2026", 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.143, 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.172, 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.176 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.116, 2026-02-12T19:08:47.457 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.116, 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.116 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.187, 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.217, 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.333 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.160 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:44 2026", 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.458 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.174, 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.159, 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.150 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.146, 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.135, 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.099 2026-02-12T19:08:47.459 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.193, 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.200, 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.274 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.162 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:43 2026", 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:08:47.460 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.157, 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.162, 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.169 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.143, 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.143, 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.133 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.172, 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.202, 2026-02-12T19:08:47.461 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.320 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.144 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:43 2026", 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.147, 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.166, 2026-02-12T19:08:47.462 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.158 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.133, 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.128, 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.113 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.166, 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.276, 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.318 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.463 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.140 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:43 2026", 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.160, 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.161, 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.158 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.464 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.150, 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.127, 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.109 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.169, 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.218, 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.286 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.164 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.465 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:44 2026", 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 2, 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.156, 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.154, 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.148 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.131, 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.118, 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.096 2026-02-12T19:08:47.466 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.195, 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.199, 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.277 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.214 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:42 2026", 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:08:47.467 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.140, 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.144, 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.153 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.113, 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.113, 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.113 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.468 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.173, 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.191, 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.327 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.125 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:45 2026", 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.469 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.149, 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.135, 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.130 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.129, 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.110, 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.090 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.163, 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.167, 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.259 2026-02-12T19:08:47.470 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.139 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:42 2026", 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 0, 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 1, 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.148, 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.133, 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.139 2026-02-12T19:08:47.471 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.135, 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.096, 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.096 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.166, 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.181, 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.297 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.187 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.472 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:43 2026", 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "front", 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.125, 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.139, 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.132 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.121, 2026-02-12T19:08:47.473 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.093, 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.093 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.130, 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.204, 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.228 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.126 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:45 2026", 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.474 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 1, 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.128, 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.114, 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.119 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.107, 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.097, 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.090 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.475 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.154, 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.154, 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.188 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.107 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: { 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: "last update": "Thu Feb 12 19:08:43 2026", 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: "stale": false, 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: "from osd": 2, 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: "to osd": 0, 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: "interface": "back", 2026-02-12T19:08:47.476 INFO:tasks.workunit.client.0.trial010.stdout: "average": { 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.110, 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.119, 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.116 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: "min": { 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.106, 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.104, 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.082 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: "max": { 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: "1min": 0.116, 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: "5min": 0.186, 2026-02-12T19:08:47.477 INFO:tasks.workunit.client.0.trial010.stdout: "15min": 0.210 2026-02-12T19:08:47.478 INFO:tasks.workunit.client.0.trial010.stdout: }, 2026-02-12T19:08:47.478 INFO:tasks.workunit.client.0.trial010.stdout: "last": 0.107 2026-02-12T19:08:47.478 INFO:tasks.workunit.client.0.trial010.stdout: } 2026-02-12T19:08:47.478 INFO:tasks.workunit.client.0.trial010.stdout: ] 2026-02-12T19:08:47.478 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:08:47.479 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:08:47.479 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: jq '.entries | length' 2026-02-12T19:08:47.479 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:96: TEST_network_ping_test1: test 12 = 12 2026-02-12T19:08:47.479 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:08:47.479 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: jq .threshold 2026-02-12T19:08:47.483 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:97: TEST_network_ping_test1: test 0 = 0 2026-02-12T19:08:47.483 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:08:47.484 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: get_asok_path mgr.x 2026-02-12T19:08:47.485 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mgr.x 2026-02-12T19:08:47.485 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mgr.x ']' 2026-02-12T19:08:47.485 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:08:47.485 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:47.485 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:47.485 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-mgr.x.asok 2026-02-12T19:08:47.486 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:08:47.486 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:100: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-mgr.x.asok dump_osd_network 99 2026-02-12T19:08:47.532 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:08:47.532 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 99, 2026-02-12T19:08:47.532 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [] 2026-02-12T19:08:47.532 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:08:47.539 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:08:47.539 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: jq .threshold 2026-02-12T19:08:47.548 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:101: TEST_network_ping_test1: test 99 = 99 2026-02-12T19:08:47.549 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: tee td/network-ping/json 2026-02-12T19:08:47.549 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: get_asok_path osd.0 2026-02-12T19:08:47.550 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:08:47.550 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:08:47.551 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:08:47.551 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:47.551 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:47.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-osd.0.asok 2026-02-12T19:08:47.551 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: CEPH_ARGS= 2026-02-12T19:08:47.551 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:102: TEST_network_ping_test1: ceph daemon /tmp/ceph-asok.19550/ceph-osd.0.asok dump_osd_network 98 2026-02-12T19:08:47.605 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:08:47.605 INFO:tasks.workunit.client.0.trial010.stdout: "threshold": 98, 2026-02-12T19:08:47.605 INFO:tasks.workunit.client.0.trial010.stdout: "entries": [] 2026-02-12T19:08:47.605 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:08:47.612 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: cat td/network-ping/json 2026-02-12T19:08:47.612 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: jq .threshold 2026-02-12T19:08:47.622 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:103: TEST_network_ping_test1: test 98 = 98 2026-02-12T19:08:47.622 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:105: TEST_network_ping_test1: rm -f td/network-ping/json 2026-02-12T19:08:47.624 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-12T19:08:47.624 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-12T19:08:47.624 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:08:47.624 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-12T19:08:47.625 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:08:47.625 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:08:47.625 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:08:47.625 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:08:47.625 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:08:47.933 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:08:47.934 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:08:47.935 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:08:47.935 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:08:47.937 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:08:47.937 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:08:47.938 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:08:47.939 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:08:47.939 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:08:47.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:08:47.940 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:08:47.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:08:47.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:08:47.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-12T19:08:48.506 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:08:48.506 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:48.506 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:48.506 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19550 2026-02-12T19:08:48.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:08:48.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:08:48.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:17: run: for func in $funcs 2026-02-12T19:08:48.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:18: run: setup td/network-ping 2026-02-12T19:08:48.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/network-ping 2026-02-12T19:08:48.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/network-ping 2026-02-12T19:08:48.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-12T19:08:48.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:08:48.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-12T19:08:48.509 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:08:48.509 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:08:48.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:08:48.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:08:48.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:08:48.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:08:48.510 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:08:48.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:08:48.511 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:08:48.512 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:08:48.512 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:08:48.513 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:08:48.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:08:48.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:08:48.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:08:48.515 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:08:48.516 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:08:48.517 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:08:48.517 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-12T19:08:48.518 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:08:48.518 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:48.518 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:48.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19550 2026-02-12T19:08:48.519 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:08:48.519 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:08:48.519 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/network-ping 2026-02-12T19:08:48.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:08:48.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:48.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:48.521 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.19550 2026-02-12T19:08:48.523 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:08:48.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:08:48.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:08:48.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/network-ping 1' TERM HUP INT 2026-02-12T19:08:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:19: run: TEST_network_ping_test2 td/network-ping 2026-02-12T19:08:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:111: TEST_network_ping_test2: local dir=td/network-ping 2026-02-12T19:08:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:113: TEST_network_ping_test2: export CEPH_ARGS 2026-02-12T19:08:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: export 'EXTRA_OPTS= --mon_warn_on_slow_ping_time=0.001' 2026-02-12T19:08:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:114: TEST_network_ping_test2: EXTRA_OPTS=' --mon_warn_on_slow_ping_time=0.001' 2026-02-12T19:08:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:115: TEST_network_ping_test2: run_mon td/network-ping a 2026-02-12T19:08:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/network-ping 2026-02-12T19:08:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:08:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:08:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:08:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/network-ping/a 2026-02-12T19:08:48.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/network-ping/a --run-dir=td/network-ping 2026-02-12T19:08:48.547 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:08:48.547 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:08:48.547 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:08:48.547 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:08:48.547 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:48.547 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:48.548 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:08:48.548 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --mon_warn_on_slow_ping_time=0.001 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/network-ping/a '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --mon-cluster-log-file=td/network-ping/log --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:08:48.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:08:48.576 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:08:48.577 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:08:48.577 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:08:48.577 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:08:48.578 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:08:48.578 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:08:48.578 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:08:48.578 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:08:48.578 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:08:48.579 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:48.579 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:48.579 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-mon.a.asok 2026-02-12T19:08:48.579 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:08:48.579 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19550/ceph-mon.a.asok config get fsid 2026-02-12T19:08:48.632 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:08:48.633 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:08:48.633 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:08:48.633 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:08:48.634 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:08:48.634 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:08:48.634 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:08:48.634 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:08:48.634 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:08:48.634 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:48.635 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:48.635 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.19550/ceph-mon.a.asok 2026-02-12T19:08:48.635 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:08:48.635 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.19550/ceph-mon.a.asok config get mon_host 2026-02-12T19:08:48.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:116: TEST_network_ping_test2: run_mgr td/network-ping x 2026-02-12T19:08:48.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/network-ping 2026-02-12T19:08:48.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:08:48.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:08:48.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:08:48.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/network-ping/x 2026-02-12T19:08:48.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:08:48.800 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:08:48.800 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:08:48.800 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:08:48.800 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:08:48.800 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:48.800 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:48.800 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:08:48.801 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:08:48.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --mon_warn_on_slow_ping_time=0.001 --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/network-ping/x '--log-file=td/network-ping/$name.log' '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --run-dir=td/network-ping '--pid-file=td/network-ping/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:08:48.822 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:117: TEST_network_ping_test2: run_osd td/network-ping 0 2026-02-12T19:08:48.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-12T19:08:48.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:08:48.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:08:48.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:08:48.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/0 2026-02-12T19:08:48.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-12T19:08:48.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:08:48.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:08:48.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:08:48.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/0' 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/0/journal' 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:08:48.824 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:08:48.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/0 2026-02-12T19:08:48.826 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:08:48.826 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8576d440-1991-4351-b672-2997255b88e3 2026-02-12T19:08:48.826 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 8576d440-1991-4351-b672-2997255b88e3 2026-02-12T19:08:48.827 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8576d440-1991-4351-b672-2997255b88e3' 2026-02-12T19:08:48.827 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:08:48.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBAJY5pRhCaMRAAjDqEwVRkRwc7uuHjqzRlOg== 2026-02-12T19:08:48.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBAJY5pRhCaMRAAjDqEwVRkRwc7uuHjqzRlOg=="}' 2026-02-12T19:08:48.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8576d440-1991-4351-b672-2997255b88e3 -i td/network-ping/0/new.json 2026-02-12T19:08:48.949 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:08:48.957 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/0/new.json 2026-02-12T19:08:48.958 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBAJY5pRhCaMRAAjDqEwVRkRwc7uuHjqzRlOg== --osd-uuid 8576d440-1991-4351-b672-2997255b88e3 2026-02-12T19:08:48.974 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:48.966+0000 7f2f537e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:48.976 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:48.970+0000 7f2f537e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:48.977 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:48.970+0000 7f2f537e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:48.978 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:48.970+0000 7f2f537e7a40 -1 bdev(0x5604bbd84400 td/network-ping/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:08:48.978 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:48.970+0000 7f2f537e7a40 -1 bluestore(td/network-ping/0) _read_fsid unparsable uuid 2026-02-12T19:08:49.726 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/0/keyring 2026-02-12T19:08:49.726 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:08:49.727 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:08:49.727 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:08:49.728 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:08:49.843 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:08:49.844 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:08:49.844 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/0 --osd-journal=td/network-ping/0/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:08:49.844 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:08:49.845 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:08:49.845 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:08:49.860 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:49.854+0000 7f6eeedffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:49.862 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:49.854+0000 7f6eeedffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:49.863 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:49.854+0000 7f6eeedffa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:49.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:08:49.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:08:49.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:08:49.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:08:49.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:08:49.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:08:49.955 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:08:49.955 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:08:49.955 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:08:49.955 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:08:50.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:08:50.413 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:50.406+0000 7f6eeedffa40 -1 Falling back to public interface 2026-02-12T19:08:51.077 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:08:51.077 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:08:51.077 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:08:51.077 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:08:51.077 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:08:51.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:08:51.217 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:51.210+0000 7f6eeedffa40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:08:51.283 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:08:52.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:08:52.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:08:52.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:08:52.285 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:08:52.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:08:52.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:08:52.494 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2732982788,v1:127.0.0.1:6801/2732982788] [v2:127.0.0.1:6802/2732982788,v1:127.0.0.1:6803/2732982788] exists,up 8576d440-1991-4351-b672-2997255b88e3 2026-02-12T19:08:52.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:08:52.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:08:52.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:118: TEST_network_ping_test2: run_osd td/network-ping 1 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/1 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/1' 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/1/journal' 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:52.495 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:08:52.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/1 2026-02-12T19:08:52.497 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:08:52.498 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=790e6104-e1d6-4b65-b296-11782e4ba156 2026-02-12T19:08:52.498 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 790e6104-e1d6-4b65-b296-11782e4ba156' 2026-02-12T19:08:52.498 INFO:tasks.workunit.client.0.trial010.stdout:add osd1 790e6104-e1d6-4b65-b296-11782e4ba156 2026-02-12T19:08:52.499 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:08:52.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBEJY5pYE1IHhAAodNICuyBVnzvLavUqVsx8Q== 2026-02-12T19:08:52.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBEJY5pYE1IHhAAodNICuyBVnzvLavUqVsx8Q=="}' 2026-02-12T19:08:52.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 790e6104-e1d6-4b65-b296-11782e4ba156 -i td/network-ping/1/new.json 2026-02-12T19:08:52.730 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:08:52.739 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/1/new.json 2026-02-12T19:08:52.740 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBEJY5pYE1IHhAAodNICuyBVnzvLavUqVsx8Q== --osd-uuid 790e6104-e1d6-4b65-b296-11782e4ba156 2026-02-12T19:08:52.757 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:52.750+0000 7f0501dfea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:52.758 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:52.750+0000 7f0501dfea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:52.759 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:52.754+0000 7f0501dfea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:52.760 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:52.754+0000 7f0501dfea40 -1 bdev(0x55b1c3402400 td/network-ping/1/block) open stat got: (1) Operation not permitted 2026-02-12T19:08:52.760 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:52.754+0000 7f0501dfea40 -1 bluestore(td/network-ping/1) _read_fsid unparsable uuid 2026-02-12T19:08:53.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/1/keyring 2026-02-12T19:08:53.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:08:53.411 INFO:tasks.workunit.client.0.trial010.stdout:adding osd1 key to auth repository 2026-02-12T19:08:53.411 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T19:08:53.411 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:08:53.705 INFO:tasks.workunit.client.0.trial010.stdout:start osd.1 2026-02-12T19:08:53.706 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T19:08:53.706 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/1 --osd-journal=td/network-ping/1/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:08:53.706 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:08:53.706 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:08:53.706 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:08:53.730 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:53.722+0000 7f556196ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:53.732 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:53.726+0000 7f556196ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:53.733 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:53.726+0000 7f556196ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:53.920 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T19:08:53.920 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:08:53.920 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T19:08:53.920 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:08:53.920 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:08:53.920 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:08:53.920 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:08:53.920 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:08:53.920 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:08:53.920 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:08:54.109 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:54.102+0000 7f556196ca40 -1 Falling back to public interface 2026-02-12T19:08:54.126 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:08:54.745 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:54.738+0000 7f556196ca40 -1 osd.1 0 log_to_monitors true 2026-02-12T19:08:55.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:08:55.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:08:55.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:08:55.127 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:08:55.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:08:55.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:08:55.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:08:56.349 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:08:56.349 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:08:56.349 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:08:56.350 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:08:56.350 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:08:56.350 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:08:56.558 INFO:tasks.workunit.client.0.trial010.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1707678003,v1:127.0.0.1:6811/1707678003] [v2:127.0.0.1:6812/1707678003,v1:127.0.0.1:6813/1707678003] exists,up 790e6104-e1d6-4b65-b296-11782e4ba156 2026-02-12T19:08:56.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:08:56.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:08:56.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:08:56.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:119: TEST_network_ping_test2: run_osd td/network-ping 2 2026-02-12T19:08:56.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/network-ping 2026-02-12T19:08:56.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:08:56.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T19:08:56.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:08:56.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/network-ping/2 2026-02-12T19:08:56.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 ' 2026-02-12T19:08:56.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/network-ping/2' 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/network-ping/2/journal' 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+=' --mon_warn_on_slow_ping_time=0.001' 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/network-ping' 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:08:56.562 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/network-ping/$name.log' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/network-ping/$name.pid' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:08:56.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:08:56.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/network-ping/2 2026-02-12T19:08:56.564 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:08:56.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=17abcfca-0c0c-471a-9878-5475c5491af7 2026-02-12T19:08:56.565 INFO:tasks.workunit.client.0.trial010.stdout:add osd2 17abcfca-0c0c-471a-9878-5475c5491af7 2026-02-12T19:08:56.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 17abcfca-0c0c-471a-9878-5475c5491af7' 2026-02-12T19:08:56.565 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:08:56.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBIJY5pP00XIhAAQr/DlKDu4SlrUqsTaIo1WA== 2026-02-12T19:08:56.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBIJY5pP00XIhAAQr/DlKDu4SlrUqsTaIo1WA=="}' 2026-02-12T19:08:56.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 17abcfca-0c0c-471a-9878-5475c5491af7 -i td/network-ping/2/new.json 2026-02-12T19:08:56.795 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:08:56.804 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/network-ping/2/new.json 2026-02-12T19:08:56.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBIJY5pP00XIhAAQr/DlKDu4SlrUqsTaIo1WA== --osd-uuid 17abcfca-0c0c-471a-9878-5475c5491af7 2026-02-12T19:08:56.821 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:56.814+0000 7f303d668a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:56.823 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:56.818+0000 7f303d668a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:56.824 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:56.818+0000 7f303d668a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:56.825 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:56.818+0000 7f303d668a40 -1 bdev(0x55b20fd32400 td/network-ping/2/block) open stat got: (1) Operation not permitted 2026-02-12T19:08:56.825 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:56.818+0000 7f303d668a40 -1 bluestore(td/network-ping/2) _read_fsid unparsable uuid 2026-02-12T19:08:57.482 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/network-ping/2/keyring 2026-02-12T19:08:57.482 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:08:57.483 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T19:08:57.483 INFO:tasks.workunit.client.0.trial010.stdout:adding osd2 key to auth repository 2026-02-12T19:08:57.483 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/network-ping/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:08:57.788 INFO:tasks.workunit.client.0.trial010.stdout:start osd.2 2026-02-12T19:08:57.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T19:08:57.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8d31c009-bd6d-4213-b238-204a9621e579 --auth-supported=none --mon-host=127.0.0.1:7146 --debug_disable_randomized_ping=true --debug_heartbeat_testing_span=5 --osd_heartbeat_interval=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/network-ping/2 --osd-journal=td/network-ping/2/journal --chdir= --mon_warn_on_slow_ping_time=0.001 --run-dir=td/network-ping '--admin-socket=/tmp/ceph-asok.19550/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/network-ping/$name.log' '--pid-file=td/network-ping/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:08:57.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:08:57.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:08:57.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:08:57.805 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:57.798+0000 7f389a1dea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:57.807 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:57.798+0000 7f389a1dea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:57.807 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:57.802+0000 7f389a1dea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:08:58.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T19:08:58.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:08:58.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T19:08:58.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:08:58.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:08:58.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:08:58.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:08:58.009 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:08:58.009 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:08:58.009 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T19:08:58.197 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:58.190+0000 7f389a1dea40 -1 Falling back to public interface 2026-02-12T19:08:58.224 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:08:58.860 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:08:58.854+0000 7f389a1dea40 -1 osd.2 0 log_to_monitors true 2026-02-12T19:08:59.226 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:08:59.226 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:08:59.227 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:08:59.227 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:08:59.227 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:08:59.227 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T19:08:59.447 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:09:00.448 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:09:00.449 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:09:00.449 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:09:00.450 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:09:00.450 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:09:00.450 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T19:09:00.660 INFO:tasks.workunit.client.0.trial010.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3398647781,v1:127.0.0.1:6819/3398647781] [v2:127.0.0.1:6820/3398647781,v1:127.0.0.1:6821/3398647781] exists,up 17abcfca-0c0c-471a-9878-5475c5491af7 2026-02-12T19:09:00.661 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:09:00.661 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:09:00.661 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:09:00.662 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:121: TEST_network_ping_test2: sleep 5 2026-02-12T19:09:05.662 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:122: TEST_network_ping_test2: ceph osd crush add-bucket dc1 datacenter 2026-02-12T19:09:05.923 INFO:tasks.workunit.client.0.trial010.stderr:bucket 'dc1' already exists 2026-02-12T19:09:05.933 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:123: TEST_network_ping_test2: ceph osd crush add-bucket dc2 datacenter 2026-02-12T19:09:06.196 INFO:tasks.workunit.client.0.trial010.stderr:bucket 'dc2' already exists 2026-02-12T19:09:06.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:124: TEST_network_ping_test2: ceph osd crush add-bucket dc3 datacenter 2026-02-12T19:09:06.483 INFO:tasks.workunit.client.0.trial010.stderr:bucket 'dc3' already exists 2026-02-12T19:09:06.492 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:125: TEST_network_ping_test2: ceph osd crush add-bucket rack1 rack 2026-02-12T19:09:06.768 INFO:tasks.workunit.client.0.trial010.stderr:bucket 'rack1' already exists 2026-02-12T19:09:06.778 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:126: TEST_network_ping_test2: ceph osd crush add-bucket rack2 rack 2026-02-12T19:09:07.052 INFO:tasks.workunit.client.0.trial010.stderr:bucket 'rack2' already exists 2026-02-12T19:09:07.062 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:127: TEST_network_ping_test2: ceph osd crush add-bucket rack3 rack 2026-02-12T19:09:07.360 INFO:tasks.workunit.client.0.trial010.stderr:bucket 'rack3' already exists 2026-02-12T19:09:07.370 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:128: TEST_network_ping_test2: ceph osd crush add-bucket host1 host 2026-02-12T19:09:07.639 INFO:tasks.workunit.client.0.trial010.stderr:bucket 'host1' already exists 2026-02-12T19:09:07.649 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:129: TEST_network_ping_test2: ceph osd crush add-bucket host2 host 2026-02-12T19:09:07.921 INFO:tasks.workunit.client.0.trial010.stderr:bucket 'host2' already exists 2026-02-12T19:09:07.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:130: TEST_network_ping_test2: ceph osd crush add-bucket host3 host 2026-02-12T19:09:08.203 INFO:tasks.workunit.client.0.trial010.stderr:bucket 'host3' already exists 2026-02-12T19:09:08.212 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:131: TEST_network_ping_test2: ceph osd crush move dc1 root=default 2026-02-12T19:09:08.472 INFO:tasks.workunit.client.0.trial010.stderr:no need to move item id -5 name 'dc1' to location {root=default} in crush map 2026-02-12T19:09:08.482 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:132: TEST_network_ping_test2: ceph osd crush move dc2 root=default 2026-02-12T19:09:08.780 INFO:tasks.workunit.client.0.trial010.stderr:no need to move item id -6 name 'dc2' to location {root=default} in crush map 2026-02-12T19:09:08.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:133: TEST_network_ping_test2: ceph osd crush move dc3 root=default 2026-02-12T19:09:09.063 INFO:tasks.workunit.client.0.trial010.stderr:no need to move item id -7 name 'dc3' to location {root=default} in crush map 2026-02-12T19:09:09.073 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:134: TEST_network_ping_test2: ceph osd crush move rack1 datacenter=dc1 2026-02-12T19:09:09.344 INFO:tasks.workunit.client.0.trial010.stderr:no need to move item id -8 name 'rack1' to location {datacenter=dc1} in crush map 2026-02-12T19:09:09.354 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:135: TEST_network_ping_test2: ceph osd crush move rack2 datacenter=dc2 2026-02-12T19:09:09.629 INFO:tasks.workunit.client.0.trial010.stderr:no need to move item id -9 name 'rack2' to location {datacenter=dc2} in crush map 2026-02-12T19:09:09.638 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:136: TEST_network_ping_test2: ceph osd crush move rack3 datacenter=dc3 2026-02-12T19:09:09.909 INFO:tasks.workunit.client.0.trial010.stderr:no need to move item id -10 name 'rack3' to location {datacenter=dc3} in crush map 2026-02-12T19:09:09.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:137: TEST_network_ping_test2: ceph osd crush move host1 rack=rack1 2026-02-12T19:09:10.186 INFO:tasks.workunit.client.0.trial010.stderr:no need to move item id -11 name 'host1' to location {rack=rack1} in crush map 2026-02-12T19:09:10.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:138: TEST_network_ping_test2: ceph osd crush move host2 rack=rack2 2026-02-12T19:09:10.468 INFO:tasks.workunit.client.0.trial010.stderr:no need to move item id -12 name 'host2' to location {rack=rack2} in crush map 2026-02-12T19:09:10.477 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:139: TEST_network_ping_test2: ceph osd crush move host3 rack=rack3 2026-02-12T19:09:10.753 INFO:tasks.workunit.client.0.trial010.stderr:no need to move item id -13 name 'host3' to location {rack=rack3} in crush map 2026-02-12T19:09:10.762 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:140: TEST_network_ping_test2: ceph osd crush set osd.0 1.0 host=host1 2026-02-12T19:09:11.020 INFO:tasks.workunit.client.0.trial010.stderr:set item id 0 name 'osd.0' weight 1 at location {host=host1}: no change 2026-02-12T19:09:11.030 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:141: TEST_network_ping_test2: ceph osd crush set osd.1 1.0 host=host2 2026-02-12T19:09:11.295 INFO:tasks.workunit.client.0.trial010.stderr:set item id 1 name 'osd.1' weight 1 at location {host=host2}: no change 2026-02-12T19:09:11.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:142: TEST_network_ping_test2: ceph osd crush set osd.2 1.0 host=host3 2026-02-12T19:09:11.573 INFO:tasks.workunit.client.0.trial010.stderr:set item id 2 name 'osd.2' weight 1 at location {host=host3}: no change 2026-02-12T19:09:11.584 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:143: TEST_network_ping_test2: ceph osd crush rule create-simple myrule default host firstn 2026-02-12T19:09:11.848 INFO:tasks.workunit.client.0.trial010.stderr:rule myrule already exists 2026-02-12T19:09:11.858 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:145: TEST_network_ping_test2: create_pool foo 16 16 replicated myrule 2026-02-12T19:09:11.858 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 16 replicated myrule 2026-02-12T19:09:12.147 INFO:tasks.workunit.client.0.trial010.stderr:pool 'foo' already exists 2026-02-12T19:09:12.157 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:09:13.159 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:148: TEST_network_ping_test2: timeout 20 rados bench -p foo 10 write -b 4096 --no-cleanup 2026-02-12T19:09:13.180 INFO:tasks.workunit.client.0.trial010.stdout:hints = 1 2026-02-12T19:09:13.180 INFO:tasks.workunit.client.0.trial010.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 10 seconds or 0 objects 2026-02-12T19:09:13.180 INFO:tasks.workunit.client.0.trial010.stdout:Object prefix: benchmark_data_trial010_31293 2026-02-12T19:09:13.180 INFO:tasks.workunit.client.0.trial010.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T19:09:13.181 INFO:tasks.workunit.client.0.trial010.stdout: 0 0 0 0 0 0 - 0 2026-02-12T19:09:14.180 INFO:tasks.workunit.client.0.trial010.stdout: 1 16 10935 10919 42.6487 42.6523 0.00115157 0.00146299 2026-02-12T19:09:15.180 INFO:tasks.workunit.client.0.trial010.stdout: 2 16 22271 22255 43.4632 44.2812 0.00110043 0.00143599 2026-02-12T19:09:16.180 INFO:tasks.workunit.client.0.trial010.stdout: 3 16 33363 33347 43.4169 43.3281 0.00107581 0.00143776 2026-02-12T19:09:17.180 INFO:tasks.workunit.client.0.trial010.stdout: 4 16 44686 44670 43.6194 44.2305 0.000871905 0.00143126 2026-02-12T19:09:18.180 INFO:tasks.workunit.client.0.trial010.stdout: 5 16 56052 56036 43.7745 44.3984 0.000954079 0.00142616 2026-02-12T19:09:19.180 INFO:tasks.workunit.client.0.trial010.stdout: 6 16 67344 67328 43.8297 44.1094 0.00159878 0.00142441 2026-02-12T19:09:20.180 INFO:tasks.workunit.client.0.trial010.stdout: 7 16 79031 79015 44.0896 45.6523 0.00130907 0.00141602 2026-02-12T19:09:21.180 INFO:tasks.workunit.client.0.trial010.stdout: 8 16 89212 89196 43.5492 39.7695 0.00127245 0.00143362 2026-02-12T19:09:22.180 INFO:tasks.workunit.client.0.trial010.stdout: 9 16 98902 98886 42.9158 37.8516 0.00122444 0.0014548 2026-02-12T19:09:23.180 INFO:tasks.workunit.client.0.trial010.stdout: 10 4 108438 108434 42.3537 37.2969 0.00137898 0.00147419 2026-02-12T19:09:24.180 INFO:tasks.workunit.client.0.trial010.stdout:Total time run: 10.0012 2026-02-12T19:09:24.180 INFO:tasks.workunit.client.0.trial010.stdout:Total writes made: 108438 2026-02-12T19:09:24.181 INFO:tasks.workunit.client.0.trial010.stdout:Write size: 4096 2026-02-12T19:09:24.181 INFO:tasks.workunit.client.0.trial010.stdout:Object size: 4096 2026-02-12T19:09:24.181 INFO:tasks.workunit.client.0.trial010.stdout:Bandwidth (MB/sec): 42.3535 2026-02-12T19:09:24.181 INFO:tasks.workunit.client.0.trial010.stdout:Stddev Bandwidth: 2.96159 2026-02-12T19:09:24.181 INFO:tasks.workunit.client.0.trial010.stdout:Max bandwidth (MB/sec): 45.6523 2026-02-12T19:09:24.181 INFO:tasks.workunit.client.0.trial010.stdout:Min bandwidth (MB/sec): 37.2969 2026-02-12T19:09:24.181 INFO:tasks.workunit.client.0.trial010.stdout:Average IOPS: 10842 2026-02-12T19:09:24.181 INFO:tasks.workunit.client.0.trial010.stdout:Stddev IOPS: 758.168 2026-02-12T19:09:24.181 INFO:tasks.workunit.client.0.trial010.stdout:Max IOPS: 11687 2026-02-12T19:09:24.181 INFO:tasks.workunit.client.0.trial010.stdout:Min IOPS: 9548 2026-02-12T19:09:24.181 INFO:tasks.workunit.client.0.trial010.stdout:Average Latency(s): 0.00147418 2026-02-12T19:09:24.182 INFO:tasks.workunit.client.0.trial010.stdout:Stddev Latency(s): 0.000667765 2026-02-12T19:09:24.182 INFO:tasks.workunit.client.0.trial010.stdout:Max latency(s): 0.00857621 2026-02-12T19:09:24.182 INFO:tasks.workunit.client.0.trial010.stdout:Min latency(s): 0.000456868 2026-02-12T19:09:24.185 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:151: TEST_network_ping_test2: sleep 10 2026-02-12T19:09:34.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:152: TEST_network_ping_test2: flush_pg_stats 2026-02-12T19:09:34.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:09:34.187 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:09:34.400 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T19:09:34.400 INFO:tasks.workunit.client.0.trial010.stderr:1 2026-02-12T19:09:34.400 INFO:tasks.workunit.client.0.trial010.stderr:2' 2026-02-12T19:09:34.400 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:09:34.400 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:09:34.400 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:09:34.482 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-12T19:09:34.482 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-12T19:09:34.483 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836491' 2026-02-12T19:09:34.483 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:09:34.483 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T19:09:34.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705674 2026-02-12T19:09:34.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705674 2026-02-12T19:09:34.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836491 1-38654705674' 2026-02-12T19:09:34.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:09:34.564 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T19:09:34.644 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-12T19:09:34.644 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-12T19:09:34.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-60129542153' 2026-02-12T19:09:34.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:09:34.645 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836491 2026-02-12T19:09:34.645 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:09:34.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:09:34.647 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-12T19:09:34.647 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:09:34.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836491 2026-02-12T19:09:34.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-12T19:09:34.648 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836491 2026-02-12T19:09:34.649 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:09:34.874 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836489 -lt 21474836491 2026-02-12T19:09:34.874 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:09:35.875 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:09:35.876 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:09:36.093 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-12T19:09:36.093 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:09:36.093 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-38654705674 2026-02-12T19:09:36.094 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:09:36.094 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T19:09:36.095 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705674 2026-02-12T19:09:36.095 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:09:36.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=38654705674 2026-02-12T19:09:36.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-12T19:09:36.097 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.1 seq 38654705674 2026-02-12T19:09:36.097 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T19:09:36.320 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-12T19:09:36.320 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:09:36.321 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-60129542153 2026-02-12T19:09:36.321 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:09:36.322 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T19:09:36.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-12T19:09:36.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:09:36.324 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=60129542153 2026-02-12T19:09:36.324 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.2 seq 60129542153 2026-02-12T19:09:36.324 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-12T19:09:36.325 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T19:09:36.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-12T19:09:36.547 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: ceph health 2026-02-12T19:09:36.547 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:154: TEST_network_ping_test2: tee td/network-ping/health 2026-02-12T19:09:36.801 INFO:tasks.workunit.client.0.trial010.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.305ms); Slow OSD heartbeats on front (longest 0.355ms) 2026-02-12T19:09:36.811 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:155: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats' td/network-ping/health 2026-02-12T19:09:36.812 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: ceph health detail 2026-02-12T19:09:36.812 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:157: TEST_network_ping_test2: tee td/network-ping/health 2026-02-12T19:09:37.101 INFO:tasks.workunit.client.0.trial010.stdout:HEALTH_WARN Slow OSD heartbeats on back (longest 0.300ms); Slow OSD heartbeats on front (longest 0.347ms) 2026-02-12T19:09:37.101 INFO:tasks.workunit.client.0.trial010.stdout:[WRN] OSD_SLOW_PING_TIME_BACK: Slow OSD heartbeats on back (longest 0.300ms) 2026-02-12T19:09:37.101 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.300 msec 2026-02-12T19:09:37.102 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.212 msec 2026-02-12T19:09:37.102 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.192 msec possibly improving 2026-02-12T19:09:37.102 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on back from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.182 msec 2026-02-12T19:09:37.102 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on back from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.170 msec possibly improving 2026-02-12T19:09:37.102 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on back from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.168 msec possibly improving 2026-02-12T19:09:37.102 INFO:tasks.workunit.client.0.trial010.stdout:[WRN] OSD_SLOW_PING_TIME_FRONT: Slow OSD heartbeats on front (longest 0.347ms) 2026-02-12T19:09:37.102 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.0 [dc1,rack1] 0.347 msec 2026-02-12T19:09:37.102 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.1 [dc2,rack2] 0.312 msec 2026-02-12T19:09:37.102 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on front from osd.0 [dc1,rack1] to osd.2 [dc3,rack3] 0.239 msec 2026-02-12T19:09:37.103 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.0 [dc1,rack1] 0.226 msec 2026-02-12T19:09:37.103 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on front from osd.1 [dc2,rack2] to osd.2 [dc3,rack3] 0.225 msec 2026-02-12T19:09:37.103 INFO:tasks.workunit.client.0.trial010.stdout: Slow OSD heartbeats on front from osd.2 [dc3,rack3] to osd.1 [dc2,rack2] 0.176 msec 2026-02-12T19:09:37.110 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:158: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_BACK td/network-ping/health 2026-02-12T19:09:37.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:159: TEST_network_ping_test2: grep -q OSD_SLOW_PING_TIME_FRONT td/network-ping/health 2026-02-12T19:09:37.113 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:160: TEST_network_ping_test2: grep -q 'Slow OSD heartbeats on front from osd[.][0-2] [[]dc[1-3],rack[1-3][]] to osd[.][0-2] [[]dc[1-3],rack[1-3][]]' td/network-ping/health 2026-02-12T19:09:37.114 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:162: TEST_network_ping_test2: rm -f td/network-ping/health 2026-02-12T19:09:37.115 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/network-ping.sh:20: run: teardown td/network-ping 2026-02-12T19:09:37.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-12T19:09:37.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:09:37.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-12T19:09:37.117 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:09:37.117 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:09:37.117 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:09:37.117 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:09:37.117 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:09:37.426 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:09:37.427 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:09:37.428 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:09:37.429 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:09:37.430 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:09:37.431 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:09:37.431 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:09:37.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:09:37.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:09:37.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:09:37.433 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:09:37.434 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:09:37.435 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:09:37.435 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-12T19:09:38.130 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:09:38.130 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:38.130 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:09:38.130 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19550 2026-02-12T19:09:38.132 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:09:38.132 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:09:38.132 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-12T19:09:38.133 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/network-ping 0 2026-02-12T19:09:38.133 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/network-ping 2026-02-12T19:09:38.133 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-12T19:09:38.133 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/network-ping KILL 2026-02-12T19:09:38.133 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:09:38.133 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:09:38.133 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:09:38.133 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:09:38.133 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:09:38.134 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:09:38.135 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:09:38.135 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:09:38.136 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:09:38.136 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:09:38.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:09:38.137 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:09:38.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:09:38.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:09:38.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:09:38.138 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:09:38.139 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:09:38.140 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-12T19:09:38.140 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/network-ping 2026-02-12T19:09:38.141 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:09:38.142 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:38.142 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.19550 2026-02-12T19:09:38.142 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.19550 2026-02-12T19:09:38.143 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:09:38.143 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:09:38.143 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-12T19:09:38.144 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-12T19:09:38.144 DEBUG:teuthology.orchestra.run.trial010:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-12T19:09:38.194 INFO:tasks.workunit:Running workunit misc/ok-to-stop.sh... 2026-02-12T19:09:38.194 DEBUG:teuthology.orchestra.run.trial010:workunit test misc/ok-to-stop.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh 2026-02-12T19:09:38.247 INFO:tasks.workunit.client.0.trial010.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-12T19:09:38.252 INFO:tasks.workunit.client.0.trial010.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-12T19:09:38.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-12T19:09:38.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-12T19:09:38.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-12T19:09:38.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-12T19:09:38.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-12T19:09:38.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-12T19:09:38.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-12T19:09:38.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-12T19:09:38.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/ok-to-stop 2026-02-12T19:09:38.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:6: run: local dir=td/ok-to-stop 2026-02-12T19:09:38.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:7: run: shift 2026-02-12T19:09:38.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: export CEPH_MON_A=127.0.0.1:7150 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:9: run: CEPH_MON_A=127.0.0.1:7150 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: export CEPH_MON_B=127.0.0.1:7151 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:10: run: CEPH_MON_B=127.0.0.1:7151 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: export CEPH_MON_C=127.0.0.1:7152 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:11: run: CEPH_MON_C=127.0.0.1:7152 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: export CEPH_MON_D=127.0.0.1:7153 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:12: run: CEPH_MON_D=127.0.0.1:7153 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: export CEPH_MON_E=127.0.0.1:7154 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:13: run: CEPH_MON_E=127.0.0.1:7154 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:14: run: export CEPH_ARGS 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: uuidgen 2026-02-12T19:09:38.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:15: run: CEPH_ARGS+='--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none ' 2026-02-12T19:09:38.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: export 'ORIG_CEPH_ARGS=--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none ' 2026-02-12T19:09:38.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:16: run: ORIG_CEPH_ARGS='--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none ' 2026-02-12T19:09:38.256 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: set 2026-02-12T19:09:38.257 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-12T19:09:38.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:18: run: local 'funcs=TEST_0_mds 2026-02-12T19:09:38.257 INFO:tasks.workunit.client.0.trial010.stderr:TEST_0_osd 2026-02-12T19:09:38.257 INFO:tasks.workunit.client.0.trial010.stderr:TEST_1_mon_checks 2026-02-12T19:09:38.257 INFO:tasks.workunit.client.0.trial010.stderr:TEST_2_mons_checks 2026-02-12T19:09:38.257 INFO:tasks.workunit.client.0.trial010.stderr:TEST_3_mons_checks 2026-02-12T19:09:38.257 INFO:tasks.workunit.client.0.trial010.stderr:TEST_4_mons_checks 2026-02-12T19:09:38.257 INFO:tasks.workunit.client.0.trial010.stderr:TEST_5_mons_checks' 2026-02-12T19:09:38.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-12T19:09:38.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-12T19:09:38.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-12T19:09:38.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-12T19:09:38.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:09:38.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:09:38.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:09:38.258 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:09:38.258 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:09:38.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:09:38.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:09:38.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:09:38.259 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:09:38.259 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:09:38.259 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:09:38.260 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:09:38.260 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:09:38.260 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:09:38.260 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:09:38.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:09:38.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:09:38.262 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:09:38.262 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:09:38.263 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:09:38.263 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:09:38.263 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:09:38.264 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:09:38.264 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:38.264 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:38.264 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:09:38.266 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:09:38.267 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:09:38.267 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-12T19:09:38.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:09:38.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:38.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:38.268 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31636 2026-02-12T19:09:38.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:09:38.271 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-12T19:09:38.271 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-12T19:09:38.271 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:09:38.271 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-12T19:09:38.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_mds td/ok-to-stop 2026-02-12T19:09:38.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:208: TEST_0_mds: local dir=td/ok-to-stop 2026-02-12T19:09:38.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:210: TEST_0_mds: CEPH_ARGS='--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-12T19:09:38.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:212: TEST_0_mds: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-12T19:09:38.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:09:38.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:09:38.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:09:38.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:09:38.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-12T19:09:38.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-12T19:09:38.295 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:09:38.296 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:09:38.296 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:09:38.296 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:09:38.296 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:38.296 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:38.296 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:09:38.296 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-12T19:09:38.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:09:38.327 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:09:38.327 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:09:38.327 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:09:38.327 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:09:38.329 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:09:38.329 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:09:38.329 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:09:38.329 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:09:38.329 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:09:38.329 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:38.329 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:38.329 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:09:38.329 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:09:38.329 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get fsid 2026-02-12T19:09:38.385 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:09:38.386 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:09:38.386 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:09:38.386 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:09:38.386 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:09:38.386 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:09:38.386 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:09:38.386 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:09:38.387 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:09:38.387 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:38.387 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:38.387 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:09:38.387 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:09:38.388 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get mon_host 2026-02-12T19:09:38.443 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:213: TEST_0_mds: run_mgr td/ok-to-stop x 2026-02-12T19:09:38.443 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-12T19:09:38.443 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:09:38.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:09:38.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:09:38.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-12T19:09:38.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:09:38.557 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:09:38.557 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:09:38.557 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:09:38.557 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:09:38.557 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:38.557 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:38.557 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:09:38.558 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:09:38.558 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:09:38.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:214: TEST_0_mds: run_osd td/ok-to-stop 0 2026-02-12T19:09:38.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-12T19:09:38.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:09:38.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:09:38.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:09:38.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-12T19:09:38.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-12T19:09:38.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:09:38.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:09:38.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:09:38.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-12T19:09:38.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-12T19:09:38.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:09:38.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:09:38.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-12T19:09:38.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-12T19:09:38.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:09:38.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:09:38.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:09:38.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:09:38.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:09:38.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:09:38.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-12T19:09:38.581 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:09:38.581 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1784320e-c2a2-409c-aa86-617a61cce027 2026-02-12T19:09:38.581 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 1784320e-c2a2-409c-aa86-617a61cce027 2026-02-12T19:09:38.582 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1784320e-c2a2-409c-aa86-617a61cce027' 2026-02-12T19:09:38.582 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:09:38.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQByJY5peJ8HIxAAaul0InLYvafKbbjtQ092tg== 2026-02-12T19:09:38.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQByJY5peJ8HIxAAaul0InLYvafKbbjtQ092tg=="}' 2026-02-12T19:09:38.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1784320e-c2a2-409c-aa86-617a61cce027 -i td/ok-to-stop/0/new.json 2026-02-12T19:09:38.701 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:09:38.709 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-12T19:09:38.710 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQByJY5peJ8HIxAAaul0InLYvafKbbjtQ092tg== --osd-uuid 1784320e-c2a2-409c-aa86-617a61cce027 2026-02-12T19:09:38.726 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:38.723+0000 7f179206da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:38.727 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:38.723+0000 7f179206da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:38.728 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:38.723+0000 7f179206da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:38.729 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:38.723+0000 7f179206da40 -1 bdev(0x55ba2d9bc400 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:09:38.729 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:38.723+0000 7f179206da40 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-12T19:09:39.416 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-12T19:09:39.416 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:09:39.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:09:39.418 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:09:39.418 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:09:39.529 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:09:39.529 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:09:39.529 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:09:39.529 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:09:39.530 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:09:39.530 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:09:39.543 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:39.539+0000 7f0e5916ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:39.544 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:39.539+0000 7f0e5916ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:39.545 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:39.543+0000 7f0e5916ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:39.643 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:09:39.643 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:09:39.643 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:09:39.643 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:09:39.643 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:09:39.643 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:09:39.644 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:09:39.644 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:09:39.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:09:39.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:09:39.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:09:39.963 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:39.959+0000 7f0e5916ba40 -1 Falling back to public interface 2026-02-12T19:09:40.642 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:40.639+0000 7f0e5916ba40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:09:40.751 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:09:40.752 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:09:40.752 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:09:40.752 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:09:40.752 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:09:40.754 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:09:40.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:09:41.893 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:09:41.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:09:41.894 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:09:41.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:09:41.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:09:41.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:09:42.109 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3469499999,v1:127.0.0.1:6801/3469499999] [v2:127.0.0.1:6802/3469499999,v1:127.0.0.1:6803/3469499999] exists,up 1784320e-c2a2-409c-aa86-617a61cce027 2026-02-12T19:09:42.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:09:42.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:09:42.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:09:42.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:215: TEST_0_mds: run_mds td/ok-to-stop a 2026-02-12T19:09:42.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-12T19:09:42.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-12T19:09:42.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=a 2026-02-12T19:09:42.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-12T19:09:42.112 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/a 2026-02-12T19:09:42.112 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-12T19:09:42.112 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:09:42.112 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:09:42.112 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:09:42.112 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:42.112 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:42.112 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:09:42.112 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id a --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-12T19:09:42.135 INFO:tasks.workunit.client.0.trial010.stdout:starting mds.a at 2026-02-12T19:09:42.139 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:217: TEST_0_mds: ceph osd pool create meta 1 2026-02-12T19:09:42.420 INFO:tasks.workunit.client.0.trial010.stderr:pool 'meta' already exists 2026-02-12T19:09:42.429 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:218: TEST_0_mds: ceph osd pool create data 1 2026-02-12T19:09:42.721 INFO:tasks.workunit.client.0.trial010.stderr:pool 'data' already exists 2026-02-12T19:09:42.732 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:219: TEST_0_mds: ceph fs new myfs meta data 2026-02-12T19:09:43.078 INFO:tasks.workunit.client.0.trial010.stderr:filesystem 'myfs' already exists 2026-02-12T19:09:43.088 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:220: TEST_0_mds: sleep 5 2026-02-12T19:09:48.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:222: TEST_0_mds: ceph mds ok-to-stop a 2026-02-12T19:09:48.272 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 2026-02-12T19:09:48.276 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:223: TEST_0_mds: ceph mds ok-to-stop a dne 2026-02-12T19:09:48.460 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 2026-02-12T19:09:48.464 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:224: TEST_0_mds: ceph mds ok-to-stop dne 2026-02-12T19:09:48.747 INFO:tasks.workunit.client.0.trial010.stderr:should be safe to stop [dne] 2026-02-12T19:09:48.759 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:226: TEST_0_mds: run_mds td/ok-to-stop b 2026-02-12T19:09:48.759 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:570: run_mds: local dir=td/ok-to-stop 2026-02-12T19:09:48.759 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:571: run_mds: shift 2026-02-12T19:09:48.759 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:572: run_mds: local id=b 2026-02-12T19:09:48.760 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:573: run_mds: shift 2026-02-12T19:09:48.760 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:574: run_mds: local data=td/ok-to-stop/b 2026-02-12T19:09:48.760 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: get_asok_path 2026-02-12T19:09:48.760 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:09:48.760 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:09:48.760 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:09:48.761 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:48.761 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:48.761 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:09:48.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:577: run_mds: ceph-mds --id b --debug-mds 20 --debug-objecter 20 --debug-ms 20 --chdir= --mds-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' 2026-02-12T19:09:48.775 INFO:tasks.workunit.client.0.trial010.stdout:starting mds.b at 2026-02-12T19:09:48.779 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:227: TEST_0_mds: sleep 5 2026-02-12T19:09:53.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:229: TEST_0_mds: ceph mds ok-to-stop a 2026-02-12T19:09:54.049 INFO:tasks.workunit.client.0.trial010.stderr:should be safe to stop [a] 2026-02-12T19:09:54.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:230: TEST_0_mds: ceph mds ok-to-stop b 2026-02-12T19:09:54.330 INFO:tasks.workunit.client.0.trial010.stderr:should be safe to stop [b] 2026-02-12T19:09:54.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:231: TEST_0_mds: ceph mds ok-to-stop a b 2026-02-12T19:09:54.525 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 4137 2026-02-12T19:09:54.529 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:232: TEST_0_mds: ceph mds ok-to-stop a dne1 dne2 2026-02-12T19:09:54.814 INFO:tasks.workunit.client.0.trial010.stderr:should be safe to stop [a,dne1,dne2] 2026-02-12T19:09:54.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:233: TEST_0_mds: ceph mds ok-to-stop b dne 2026-02-12T19:09:55.118 INFO:tasks.workunit.client.0.trial010.stderr:should be safe to stop [b,dne] 2026-02-12T19:09:55.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:234: TEST_0_mds: ceph mds ok-to-stop a b dne 2026-02-12T19:09:55.332 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: insufficent standby MDS daemons to stop active gids 4123 and/or standby gids 4137 2026-02-12T19:09:55.336 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:235: TEST_0_mds: ceph mds ok-to-stop dne1 dne2 2026-02-12T19:09:55.617 INFO:tasks.workunit.client.0.trial010.stderr:should be safe to stop [dne1,dne2] 2026-02-12T19:09:55.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:237: TEST_0_mds: kill_daemons td/ok-to-stop KILL mds.a 2026-02-12T19:09:55.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:09:55.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:09:55.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:09:55.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:09:55.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:09:55.735 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:09:55.735 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-12T19:09:55.736 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:09:55.736 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:09:55.736 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:09:55.736 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:09:55.736 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:09:55.843 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:09:55.843 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-12T19:09:55.843 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:09:55.843 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:09:55.843 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:09:55.844 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:09:55.844 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:09:55.844 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:09:55.844 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:09:55.844 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:09:55.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:09:55.850 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:09:55.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:09:55.851 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:09:55.852 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:09:55.852 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:09:55.852 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:09:55.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:09:55.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:09:55.854 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:09:55.854 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:09:55.855 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:09:55.856 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:09:55.856 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:09:55.861 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:09:55.861 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:55.861 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:55.861 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:09:55.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:09:55.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:09:55.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-12T19:09:55.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-12T19:09:55.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-12T19:09:55.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-12T19:09:55.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:09:55.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:09:55.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:09:55.864 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:09:55.864 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:09:55.865 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:09:55.865 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:09:55.865 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:09:55.866 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:09:55.866 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:09:55.867 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:09:55.867 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:09:55.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:09:55.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:09:55.869 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:09:55.870 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:09:55.870 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:09:55.870 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:09:55.870 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:09:55.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:09:55.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:09:55.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:09:55.874 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:09:55.874 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:55.874 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:55.874 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:09:55.875 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:09:55.875 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:09:55.876 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-12T19:09:55.877 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:09:55.877 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:55.877 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:55.877 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31636 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_0_osd td/ok-to-stop 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:241: TEST_0_osd: local dir=td/ok-to-stop 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:243: TEST_0_osd: CEPH_ARGS='--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:245: TEST_0_osd: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-12T19:09:55.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-12T19:09:55.901 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:09:55.901 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:09:55.901 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:09:55.901 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:09:55.901 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:55.901 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:55.902 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:09:55.902 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-12T19:09:55.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:09:55.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:09:55.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:09:55.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:09:55.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:09:55.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:09:55.933 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:09:55.933 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:09:55.933 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:09:55.933 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:09:55.933 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:55.933 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:55.933 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:09:55.933 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:09:55.933 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get fsid 2026-02-12T19:09:56.007 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:09:56.007 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:09:56.007 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:09:56.007 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:09:56.007 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:09:56.007 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:09:56.007 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:09:56.007 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:09:56.008 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:09:56.008 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:56.008 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:56.008 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:09:56.008 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:09:56.008 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get mon_host 2026-02-12T19:09:56.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:246: TEST_0_osd: run_mgr td/ok-to-stop x 2026-02-12T19:09:56.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ok-to-stop 2026-02-12T19:09:56.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:09:56.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:09:56.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:09:56.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ok-to-stop/x 2026-02-12T19:09:56.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:09:56.175 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:09:56.176 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:09:56.176 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:09:56.176 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:09:56.176 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:56.176 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:56.177 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:09:56.177 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:09:56.177 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ok-to-stop/x '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:09:56.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:247: TEST_0_osd: run_osd td/ok-to-stop 0 2026-02-12T19:09:56.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-12T19:09:56.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:09:56.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:09:56.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:09:56.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/0 2026-02-12T19:09:56.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-12T19:09:56.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:09:56.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:09:56.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:09:56.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/0' 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/0/journal' 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:09:56.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:09:56.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/0 2026-02-12T19:09:56.199 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:09:56.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ae79187e-0efa-4670-b324-390ff8879c1f 2026-02-12T19:09:56.199 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 ae79187e-0efa-4670-b324-390ff8879c1f 2026-02-12T19:09:56.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ae79187e-0efa-4670-b324-390ff8879c1f' 2026-02-12T19:09:56.200 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:09:56.208 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCEJY5poz5hDBAAbqDtvzQFw7YjcXFKc0dKSQ== 2026-02-12T19:09:56.209 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCEJY5poz5hDBAAbqDtvzQFw7YjcXFKc0dKSQ=="}' 2026-02-12T19:09:56.209 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ae79187e-0efa-4670-b324-390ff8879c1f -i td/ok-to-stop/0/new.json 2026-02-12T19:09:56.321 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:09:56.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/0/new.json 2026-02-12T19:09:56.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCEJY5poz5hDBAAbqDtvzQFw7YjcXFKc0dKSQ== --osd-uuid ae79187e-0efa-4670-b324-390ff8879c1f 2026-02-12T19:09:56.346 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:56.343+0000 7f9d5e25aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:56.347 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:56.343+0000 7f9d5e25aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:56.349 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:56.343+0000 7f9d5e25aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:56.349 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:56.343+0000 7f9d5e25aa40 -1 bdev(0x556213144400 td/ok-to-stop/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:09:56.349 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:56.347+0000 7f9d5e25aa40 -1 bluestore(td/ok-to-stop/0) _read_fsid unparsable uuid 2026-02-12T19:09:57.184 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/0/keyring 2026-02-12T19:09:57.184 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:09:57.185 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:09:57.185 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:09:57.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:09:57.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:09:57.308 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:09:57.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/0 --osd-journal=td/ok-to-stop/0/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:09:57.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:09:57.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:09:57.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:09:57.323 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:57.319+0000 7f639fd5ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:57.324 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:57.319+0000 7f639fd5ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:57.325 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:57.323+0000 7f639fd5ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:09:57.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:09:57.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:09:57.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:09:57.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:09:57.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:09:57.433 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:09:57.433 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:09:57.433 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:09:57.434 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:09:57.434 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:09:57.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:09:57.847 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:57.843+0000 7f639fd5ca40 -1 Falling back to public interface 2026-02-12T19:09:58.541 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:09:58.541 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:09:58.542 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:09:58.542 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:09:58.542 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:09:58.542 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:09:58.742 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:09:58.739+0000 7f639fd5ca40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:09:58.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:09:59.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:09:59.759 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:09:59.759 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:09:59.760 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:09:59.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:09:59.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:09:59.978 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:10:00.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:10:00.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:10:00.980 INFO:tasks.workunit.client.0.trial010.stdout:3 2026-02-12T19:10:00.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-12T19:10:00.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:10:00.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:10:01.199 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1350099545,v1:127.0.0.1:6801/1350099545] [v2:127.0.0.1:6802/1350099545,v1:127.0.0.1:6803/1350099545] exists,up ae79187e-0efa-4670-b324-390ff8879c1f 2026-02-12T19:10:01.202 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:10:01.202 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:10:01.202 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:10:01.202 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:248: TEST_0_osd: run_osd td/ok-to-stop 1 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/1 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/1' 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/1/journal' 2026-02-12T19:10:01.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:10:01.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:10:01.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-12T19:10:01.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-12T19:10:01.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:10:01.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:10:01.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:10:01.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:10:01.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:10:01.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:10:01.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/1 2026-02-12T19:10:01.206 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:10:01.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=05907d4f-1733-4f19-bae5-935a7c75760b 2026-02-12T19:10:01.206 INFO:tasks.workunit.client.0.trial010.stdout:add osd1 05907d4f-1733-4f19-bae5-935a7c75760b 2026-02-12T19:10:01.207 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 05907d4f-1733-4f19-bae5-935a7c75760b' 2026-02-12T19:10:01.207 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:10:01.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCJJY5ptrfCDBAA5m6eZnu1t8ZGZFFG6qqXBA== 2026-02-12T19:10:01.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCJJY5ptrfCDBAA5m6eZnu1t8ZGZFFG6qqXBA=="}' 2026-02-12T19:10:01.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 05907d4f-1733-4f19-bae5-935a7c75760b -i td/ok-to-stop/1/new.json 2026-02-12T19:10:01.439 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:10:01.450 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/1/new.json 2026-02-12T19:10:01.451 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCJJY5ptrfCDBAA5m6eZnu1t8ZGZFFG6qqXBA== --osd-uuid 05907d4f-1733-4f19-bae5-935a7c75760b 2026-02-12T19:10:01.474 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:01.471+0000 7f130ab14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:01.476 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:01.471+0000 7f130ab14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:01.477 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:01.471+0000 7f130ab14a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:01.477 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:01.475+0000 7f130ab14a40 -1 bdev(0x564b5aa92400 td/ok-to-stop/1/block) open stat got: (1) Operation not permitted 2026-02-12T19:10:01.477 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:01.475+0000 7f130ab14a40 -1 bluestore(td/ok-to-stop/1) _read_fsid unparsable uuid 2026-02-12T19:10:02.145 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/1/keyring 2026-02-12T19:10:02.145 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:10:02.145 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T19:10:02.146 INFO:tasks.workunit.client.0.trial010.stdout:adding osd1 key to auth repository 2026-02-12T19:10:02.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:10:02.456 INFO:tasks.workunit.client.0.trial010.stdout:start osd.1 2026-02-12T19:10:02.457 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T19:10:02.457 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/1 --osd-journal=td/ok-to-stop/1/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:10:02.457 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:10:02.457 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:10:02.457 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:10:02.479 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:02.475+0000 7f3bf1f83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:02.481 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:02.475+0000 7f3bf1f83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:02.482 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:02.479+0000 7f3bf1f83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:02.671 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T19:10:02.671 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:10:02.671 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T19:10:02.671 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:10:02.671 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:10:02.671 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:10:02.671 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:10:02.672 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:10:02.672 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:10:02.672 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:10:02.823 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:02.819+0000 7f3bf1f83a40 -1 Falling back to public interface 2026-02-12T19:10:02.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:10:03.486 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:03.483+0000 7f3bf1f83a40 -1 osd.1 0 log_to_monitors true 2026-02-12T19:10:03.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:10:03.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:10:03.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:10:03.881 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:10:03.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:10:03.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:10:04.092 INFO:tasks.workunit.client.0.trial010.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3264372564,v1:127.0.0.1:6811/3264372564] [v2:127.0.0.1:6812/3264372564,v1:127.0.0.1:6813/3264372564] exists,up 05907d4f-1733-4f19-bae5-935a7c75760b 2026-02-12T19:10:04.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:10:04.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:10:04.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:10:04.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:249: TEST_0_osd: run_osd td/ok-to-stop 2 2026-02-12T19:10:04.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-12T19:10:04.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:10:04.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T19:10:04.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:10:04.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/2 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/2' 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/2/journal' 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:10:04.097 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:10:04.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:10:04.099 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:10:04.099 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:10:04.099 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:10:04.099 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/2 2026-02-12T19:10:04.099 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:10:04.099 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5021e733-457e-4746-b140-a519f2d0700d 2026-02-12T19:10:04.100 INFO:tasks.workunit.client.0.trial010.stdout:add osd2 5021e733-457e-4746-b140-a519f2d0700d 2026-02-12T19:10:04.100 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5021e733-457e-4746-b140-a519f2d0700d' 2026-02-12T19:10:04.100 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:10:04.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCMJY5p7RplBhAAc1mvsbukpS15M2rUbPbNTA== 2026-02-12T19:10:04.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCMJY5p7RplBhAAc1mvsbukpS15M2rUbPbNTA=="}' 2026-02-12T19:10:04.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5021e733-457e-4746-b140-a519f2d0700d -i td/ok-to-stop/2/new.json 2026-02-12T19:10:04.316 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:10:04.325 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/2/new.json 2026-02-12T19:10:04.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCMJY5p7RplBhAAc1mvsbukpS15M2rUbPbNTA== --osd-uuid 5021e733-457e-4746-b140-a519f2d0700d 2026-02-12T19:10:04.341 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:04.339+0000 7fcc86609a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:04.343 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:04.339+0000 7fcc86609a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:04.344 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:04.339+0000 7fcc86609a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:04.344 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:04.339+0000 7fcc86609a40 -1 bdev(0x55c61476a400 td/ok-to-stop/2/block) open stat got: (1) Operation not permitted 2026-02-12T19:10:04.344 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:04.339+0000 7fcc86609a40 -1 bluestore(td/ok-to-stop/2) _read_fsid unparsable uuid 2026-02-12T19:10:05.014 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/2/keyring 2026-02-12T19:10:05.014 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:10:05.016 INFO:tasks.workunit.client.0.trial010.stdout:adding osd2 key to auth repository 2026-02-12T19:10:05.016 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T19:10:05.016 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:10:05.304 INFO:tasks.workunit.client.0.trial010.stdout:start osd.2 2026-02-12T19:10:05.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T19:10:05.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/2 --osd-journal=td/ok-to-stop/2/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:10:05.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:10:05.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:10:05.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:10:05.326 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:05.323+0000 7f568d038a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:05.328 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:05.323+0000 7f568d038a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:05.329 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:05.327+0000 7f568d038a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:05.521 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T19:10:05.521 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:10:05.521 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T19:10:05.521 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:10:05.521 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:10:05.522 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:10:05.522 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:10:05.522 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:10:05.522 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:10:05.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T19:10:05.707 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:05.703+0000 7f568d038a40 -1 Falling back to public interface 2026-02-12T19:10:05.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:10:06.290 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:06.287+0000 7f568d038a40 -1 osd.2 0 log_to_monitors true 2026-02-12T19:10:06.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:10:06.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:10:06.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:10:06.730 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:10:06.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:10:06.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T19:10:06.940 INFO:tasks.workunit.client.0.trial010.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/264823352,v1:127.0.0.1:6819/264823352] [v2:127.0.0.1:6820/264823352,v1:127.0.0.1:6821/264823352] exists,up 5021e733-457e-4746-b140-a519f2d0700d 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:250: TEST_0_osd: run_osd td/ok-to-stop 3 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ok-to-stop 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ok-to-stop/3 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ok-to-stop/3' 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ok-to-stop/3/journal' 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ok-to-stop' 2026-02-12T19:10:06.941 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ok-to-stop/$name.log' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ok-to-stop/$name.pid' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:10:06.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ok-to-stop/3 2026-02-12T19:10:06.944 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:10:06.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6ba7abb3-d04d-48b1-859f-7ce34bcc23a4 2026-02-12T19:10:06.945 INFO:tasks.workunit.client.0.trial010.stdout:add osd3 6ba7abb3-d04d-48b1-859f-7ce34bcc23a4 2026-02-12T19:10:06.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 6ba7abb3-d04d-48b1-859f-7ce34bcc23a4' 2026-02-12T19:10:06.945 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:10:06.955 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCOJY5pnzbkOBAAPPC1SbU78q5VbiDhf8/RSQ== 2026-02-12T19:10:06.955 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCOJY5pnzbkOBAAPPC1SbU78q5VbiDhf8/RSQ=="}' 2026-02-12T19:10:06.955 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6ba7abb3-d04d-48b1-859f-7ce34bcc23a4 -i td/ok-to-stop/3/new.json 2026-02-12T19:10:07.177 INFO:tasks.workunit.client.0.trial010.stdout:3 2026-02-12T19:10:07.188 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ok-to-stop/3/new.json 2026-02-12T19:10:07.189 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCOJY5pnzbkOBAAPPC1SbU78q5VbiDhf8/RSQ== --osd-uuid 6ba7abb3-d04d-48b1-859f-7ce34bcc23a4 2026-02-12T19:10:07.211 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:07.207+0000 7ffa1c9a9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:07.213 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:07.207+0000 7ffa1c9a9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:07.214 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:07.211+0000 7ffa1c9a9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:07.214 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:07.211+0000 7ffa1c9a9a40 -1 bdev(0x556d8aebe400 td/ok-to-stop/3/block) open stat got: (1) Operation not permitted 2026-02-12T19:10:07.215 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:07.211+0000 7ffa1c9a9a40 -1 bluestore(td/ok-to-stop/3) _read_fsid unparsable uuid 2026-02-12T19:10:07.816 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ok-to-stop/3/keyring 2026-02-12T19:10:07.816 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:10:07.818 INFO:tasks.workunit.client.0.trial010.stdout:adding osd3 key to auth repository 2026-02-12T19:10:07.818 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-12T19:10:07.818 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ok-to-stop/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:10:08.116 INFO:tasks.workunit.client.0.trial010.stdout:start osd.3 2026-02-12T19:10:08.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-12T19:10:08.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ok-to-stop/3 --osd-journal=td/ok-to-stop/3/journal --chdir= --run-dir=td/ok-to-stop '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ok-to-stop/$name.log' '--pid-file=td/ok-to-stop/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:10:08.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:10:08.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:10:08.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:10:08.139 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:08.135+0000 7f2f9cc43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:08.141 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:08.135+0000 7f2f9cc43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:08.142 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:08.139+0000 7f2f9cc43a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:10:08.321 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-12T19:10:08.321 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:10:08.321 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-12T19:10:08.321 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:10:08.321 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:10:08.321 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:10:08.321 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:10:08.322 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:10:08.322 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:10:08.322 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T19:10:08.539 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:08.535+0000 7f2f9cc43a40 -1 Falling back to public interface 2026-02-12T19:10:08.543 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:10:09.163 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:10:09.159+0000 7f2f9cc43a40 -1 osd.3 0 log_to_monitors true 2026-02-12T19:10:09.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:10:09.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:10:09.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:10:09.544 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:10:09.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:10:09.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T19:10:09.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:10:10.760 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:10:10.760 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:10:10.760 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:10:10.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:10:10.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:10:10.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-12T19:10:10.976 INFO:tasks.workunit.client.0.trial010.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/582182248,v1:127.0.0.1:6827/582182248] [v2:127.0.0.1:6828/582182248,v1:127.0.0.1:6829/582182248] exists,up 6ba7abb3-d04d-48b1-859f-7ce34bcc23a4 2026-02-12T19:10:10.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:10:10.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:10:10.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:10:10.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:252: TEST_0_osd: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2026-02-12T19:10:11.234 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:253: TEST_0_osd: ceph osd pool create ec erasure ec-profile 2026-02-12T19:10:11.602 INFO:tasks.workunit.client.0.trial010.stderr:pool 'ec' already exists 2026-02-12T19:10:11.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:255: TEST_0_osd: wait_for_clean 2026-02-12T19:10:11.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:10:11.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:10:11.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:10:11.614 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:10:11.614 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:10:11.615 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:10:11.615 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:10:11.615 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:10:11.615 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:10:11.713 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:10:11.713 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:10:11.713 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:10:11.713 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:10:11.713 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:10:11.713 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:10:11.922 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T19:10:11.922 INFO:tasks.workunit.client.0.trial010.stderr:1 2026-02-12T19:10:11.922 INFO:tasks.workunit.client.0.trial010.stderr:2 2026-02-12T19:10:11.922 INFO:tasks.workunit.client.0.trial010.stderr:3' 2026-02-12T19:10:11.922 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:10:11.922 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:11.922 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:10:12.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836486 2026-02-12T19:10:12.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836486 2026-02-12T19:10:12.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836486' 2026-02-12T19:10:12.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:12.002 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T19:10:12.077 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-12T19:10:12.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-12T19:10:12.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836486 1-42949672964' 2026-02-12T19:10:12.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:12.078 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T19:10:12.157 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509443 2026-02-12T19:10:12.157 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509443 2026-02-12T19:10:12.157 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836486 1-42949672964 2-64424509443' 2026-02-12T19:10:12.157 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:12.157 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T19:10:12.239 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378627 2026-02-12T19:10:12.239 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378627 2026-02-12T19:10:12.239 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836486 1-42949672964 2-64424509443 3-81604378627' 2026-02-12T19:10:12.239 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:12.239 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836486 2026-02-12T19:10:12.239 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:12.240 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:10:12.241 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836486 2026-02-12T19:10:12.241 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:12.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836486 2026-02-12T19:10:12.243 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836486 2026-02-12T19:10:12.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836486' 2026-02-12T19:10:12.243 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:10:12.462 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836486 2026-02-12T19:10:12.462 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:12.463 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672964 2026-02-12T19:10:12.463 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:12.464 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T19:10:12.465 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-12T19:10:12.465 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:12.466 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672964 2026-02-12T19:10:12.466 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-12T19:10:12.466 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.1 seq 42949672964 2026-02-12T19:10:12.467 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T19:10:12.674 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-12T19:10:12.674 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:12.674 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-64424509443 2026-02-12T19:10:12.674 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:12.676 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T19:10:12.676 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509443 2026-02-12T19:10:12.676 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:12.678 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=64424509443 2026-02-12T19:10:12.678 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.2 seq 64424509443 2026-02-12T19:10:12.678 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 64424509443' 2026-02-12T19:10:12.678 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T19:10:12.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-12T19:10:12.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:12.899 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-81604378627 2026-02-12T19:10:12.899 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:12.900 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T19:10:12.901 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378627 2026-02-12T19:10:12.901 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:12.902 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378627 2026-02-12T19:10:12.902 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.3 seq 81604378627 2026-02-12T19:10:12.902 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 81604378627' 2026-02-12T19:10:12.902 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T19:10:13.109 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378627 -lt 81604378627 2026-02-12T19:10:13.109 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:10:13.109 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:10:13.109 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:10:13.386 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-12T19:10:13.387 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:10:13.387 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:10:13.387 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:10:13.387 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:10:13.387 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:10:13.387 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:10:13.387 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:10:13.593 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-12T19:10:13.593 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:10:13.594 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:10:13.594 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:10:13.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-12T19:10:13.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:10:13.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:10:13.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:258: TEST_0_osd: ceph osd pool set ec min_size 3 2026-02-12T19:10:14.186 INFO:tasks.workunit.client.0.trial010.stderr:set pool 1 min_size to 3 2026-02-12T19:10:14.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:259: TEST_0_osd: wait_for_clean 2026-02-12T19:10:14.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:10:14.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:10:14.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:10:14.197 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:10:14.197 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:10:14.197 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:10:14.198 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:10:14.198 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:10:14.198 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:10:14.311 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:10:14.311 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:10:14.311 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:10:14.311 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:10:14.311 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:10:14.311 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:10:14.532 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T19:10:14.532 INFO:tasks.workunit.client.0.trial010.stderr:1 2026-02-12T19:10:14.532 INFO:tasks.workunit.client.0.trial010.stderr:2 2026-02-12T19:10:14.532 INFO:tasks.workunit.client.0.trial010.stderr:3' 2026-02-12T19:10:14.532 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:10:14.532 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:14.532 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:10:14.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836488 2026-02-12T19:10:14.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836488 2026-02-12T19:10:14.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488' 2026-02-12T19:10:14.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:14.613 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T19:10:14.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672967 2026-02-12T19:10:14.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672967 2026-02-12T19:10:14.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672967' 2026-02-12T19:10:14.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:14.689 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T19:10:14.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509446 2026-02-12T19:10:14.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509446 2026-02-12T19:10:14.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672967 2-64424509446' 2026-02-12T19:10:14.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:14.766 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T19:10:14.845 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378629 2026-02-12T19:10:14.845 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378629 2026-02-12T19:10:14.845 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836488 1-42949672967 2-64424509446 3-81604378629' 2026-02-12T19:10:14.845 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:14.845 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836488 2026-02-12T19:10:14.845 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:14.846 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:10:14.847 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836488 2026-02-12T19:10:14.847 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:14.848 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836488 2026-02-12T19:10:14.848 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836488 2026-02-12T19:10:14.849 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-12T19:10:14.849 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:10:15.074 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-12T19:10:15.074 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:10:16.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:10:16.076 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:10:16.283 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-12T19:10:16.283 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:10:17.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T19:10:17.285 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:10:17.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836489 -lt 21474836488 2026-02-12T19:10:17.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:17.503 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672967 2026-02-12T19:10:17.503 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:17.504 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T19:10:17.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672967 2026-02-12T19:10:17.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:17.506 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672967 2026-02-12T19:10:17.507 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.1 seq 42949672967 2026-02-12T19:10:17.507 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-12T19:10:17.507 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T19:10:17.733 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-12T19:10:17.733 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:17.734 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-64424509446 2026-02-12T19:10:17.734 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:17.735 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T19:10:17.736 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509446 2026-02-12T19:10:17.736 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:17.737 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=64424509446 2026-02-12T19:10:17.737 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.2 seq 64424509446 2026-02-12T19:10:17.737 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 64424509446' 2026-02-12T19:10:17.737 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T19:10:17.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509446 -lt 64424509446 2026-02-12T19:10:17.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:17.957 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-81604378629 2026-02-12T19:10:17.957 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:17.958 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T19:10:17.959 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378629 2026-02-12T19:10:17.959 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:17.960 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378629 2026-02-12T19:10:17.960 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.3 seq 81604378629 2026-02-12T19:10:17.960 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 81604378629' 2026-02-12T19:10:17.961 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T19:10:18.180 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378630 -lt 81604378629 2026-02-12T19:10:18.181 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:10:18.181 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:10:18.181 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:10:18.458 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-12T19:10:18.458 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:10:18.458 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:10:18.458 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:10:18.458 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:10:18.458 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:10:18.458 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:10:18.459 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:10:18.666 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-12T19:10:18.666 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:10:18.667 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:10:18.667 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:10:18.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-12T19:10:18.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:10:18.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:10:18.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:261: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-12T19:10:19.148 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:19.157 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:262: TEST_0_osd: ceph osd ok-to-stop 1 2026-02-12T19:10:19.377 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:19.387 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:263: TEST_0_osd: ceph osd ok-to-stop 2 2026-02-12T19:10:19.606 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:19.615 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:264: TEST_0_osd: ceph osd ok-to-stop 3 2026-02-12T19:10:19.835 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:19.844 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:265: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-12T19:10:20.008 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":false,"osds":[0,1],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:20.008 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-12T19:10:20.012 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:266: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-12T19:10:20.166 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":false,"osds":[2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:20.166 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-12T19:10:20.171 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-12T19:10:20.171 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:267: TEST_0_osd: grep '[0]' 2026-02-12T19:10:20.389 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:20.389 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: ceph osd ok-to-stop 1 --max 2 2026-02-12T19:10:20.390 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:268: TEST_0_osd: grep '[1]' 2026-02-12T19:10:20.603 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:20.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:271: TEST_0_osd: ceph osd pool set ec min_size 2 2026-02-12T19:10:20.911 INFO:tasks.workunit.client.0.trial010.stderr:set pool 1 min_size to 2 2026-02-12T19:10:20.927 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:272: TEST_0_osd: wait_for_clean 2026-02-12T19:10:20.927 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:10:20.927 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:10:20.928 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:10:20.928 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:10:20.928 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:10:20.928 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:10:20.929 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:10:20.929 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:10:20.929 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:10:21.039 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:10:21.039 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:10:21.039 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:10:21.039 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:10:21.039 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:10:21.039 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:10:21.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T19:10:21.252 INFO:tasks.workunit.client.0.trial010.stderr:1 2026-02-12T19:10:21.252 INFO:tasks.workunit.client.0.trial010.stderr:2 2026-02-12T19:10:21.252 INFO:tasks.workunit.client.0.trial010.stderr:3' 2026-02-12T19:10:21.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:10:21.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:21.253 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:10:21.333 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836492 2026-02-12T19:10:21.333 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836492 2026-02-12T19:10:21.333 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492' 2026-02-12T19:10:21.333 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:21.333 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T19:10:21.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672970 2026-02-12T19:10:21.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672970 2026-02-12T19:10:21.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672970' 2026-02-12T19:10:21.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:21.410 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T19:10:21.489 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509449 2026-02-12T19:10:21.489 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509449 2026-02-12T19:10:21.489 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672970 2-64424509449' 2026-02-12T19:10:21.489 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:21.489 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T19:10:21.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378633 2026-02-12T19:10:21.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378633 2026-02-12T19:10:21.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836492 1-42949672970 2-64424509449 3-81604378633' 2026-02-12T19:10:21.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:21.566 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836492 2026-02-12T19:10:21.566 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:21.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:10:21.568 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836492 2026-02-12T19:10:21.568 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:21.569 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836492 2026-02-12T19:10:21.569 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836492 2026-02-12T19:10:21.569 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-12T19:10:21.570 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:10:21.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836489 -lt 21474836492 2026-02-12T19:10:21.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:10:22.787 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:10:22.787 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:10:22.998 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-12T19:10:22.998 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:22.998 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672970 2026-02-12T19:10:22.998 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:23.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T19:10:23.000 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672970 2026-02-12T19:10:23.000 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:23.002 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672970 2026-02-12T19:10:23.002 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.1 seq 42949672970 2026-02-12T19:10:23.002 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-12T19:10:23.002 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T19:10:23.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-12T19:10:23.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:23.220 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-64424509449 2026-02-12T19:10:23.220 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:23.222 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T19:10:23.222 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509449 2026-02-12T19:10:23.222 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:23.224 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=64424509449 2026-02-12T19:10:23.224 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.2 seq 64424509449 2026-02-12T19:10:23.224 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 64424509449' 2026-02-12T19:10:23.224 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T19:10:23.448 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509449 -lt 64424509449 2026-02-12T19:10:23.448 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:23.448 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-81604378633 2026-02-12T19:10:23.449 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:23.450 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T19:10:23.451 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378633 2026-02-12T19:10:23.451 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:23.452 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378633 2026-02-12T19:10:23.453 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.3 seq 81604378633 2026-02-12T19:10:23.453 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 81604378633' 2026-02-12T19:10:23.453 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T19:10:23.674 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378633 -lt 81604378633 2026-02-12T19:10:23.674 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:10:23.675 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:10:23.675 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:10:23.958 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-12T19:10:23.958 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:10:23.958 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:10:23.958 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:10:23.958 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:10:23.958 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:10:23.958 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:10:23.958 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:10:24.167 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=32 2026-02-12T19:10:24.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:10:24.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:10:24.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:10:24.445 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 = 32 2026-02-12T19:10:24.445 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:10:24.446 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:10:24.446 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:274: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-12T19:10:24.645 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:24.656 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:275: TEST_0_osd: ceph osd ok-to-stop 2 3 2026-02-12T19:10:24.858 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[2,3],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:24.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:276: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-12T19:10:25.022 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:25.022 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-12T19:10:25.025 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:277: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-12T19:10:25.181 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:25.182 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-12T19:10:25.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: ceph osd ok-to-stop 0 --max 2 2026-02-12T19:10:25.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:279: TEST_0_osd: grep '[0,1]' 2026-02-12T19:10:25.401 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:25.401 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: ceph osd ok-to-stop 0 --max 20 2026-02-12T19:10:25.402 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:280: TEST_0_osd: grep '[0,1]' 2026-02-12T19:10:25.620 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:25.621 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: ceph osd ok-to-stop 2 --max 2 2026-02-12T19:10:25.622 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:281: TEST_0_osd: grep '[2,3]' 2026-02-12T19:10:25.829 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:25.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: ceph osd ok-to-stop 2 --max 20 2026-02-12T19:10:25.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:282: TEST_0_osd: grep '[2,3]' 2026-02-12T19:10:26.048 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[0,2],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:26.049 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:285: TEST_0_osd: kill_daemons td/ok-to-stop TERM osd.0 2026-02-12T19:10:26.049 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:10:26.049 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:10:26.049 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:10:26.049 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:10:26.049 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:10:26.357 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:10:26.357 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:286: TEST_0_osd: ceph osd down 0 2026-02-12T19:10:26.561 INFO:tasks.workunit.client.0.trial010.stderr:osd.0 is already down. 2026-02-12T19:10:26.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:287: TEST_0_osd: wait_for_peered 2026-02-12T19:10:26.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1731: wait_for_peered: local cmd= 2026-02-12T19:10:26.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1732: wait_for_peered: local num_peered=-1 2026-02-12T19:10:26.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1733: wait_for_peered: local cur_peered 2026-02-12T19:10:26.572 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: get_timeout_delays 90 .1 2026-02-12T19:10:26.572 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:10:26.572 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:10:26.573 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:10:26.573 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:10:26.573 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:10:26.675 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:10:26.675 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1734: wait_for_peered: local -a delays 2026-02-12T19:10:26.676 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1735: wait_for_peered: local -i loop=0 2026-02-12T19:10:26.676 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1737: wait_for_peered: flush_pg_stats 2026-02-12T19:10:26.676 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:10:26.676 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:10:26.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T19:10:26.887 INFO:tasks.workunit.client.0.trial010.stderr:1 2026-02-12T19:10:26.887 INFO:tasks.workunit.client.0.trial010.stderr:2 2026-02-12T19:10:26.887 INFO:tasks.workunit.client.0.trial010.stderr:3' 2026-02-12T19:10:26.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:10:26.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:26.888 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:10:26.949 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-12T19:10:26.949 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-12T19:10:26.949 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-12T19:10:26.949 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-12T19:10:26.950 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:26.950 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T19:10:27.027 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672973 2026-02-12T19:10:27.028 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672973 2026-02-12T19:10:27.028 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-42949672973' 2026-02-12T19:10:27.028 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:27.028 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.2 flush_pg_stats 2026-02-12T19:10:27.105 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509452 2026-02-12T19:10:27.105 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509452 2026-02-12T19:10:27.105 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-42949672973 2-64424509452' 2026-02-12T19:10:27.106 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:10:27.106 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.3 flush_pg_stats 2026-02-12T19:10:27.182 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378636 2026-02-12T19:10:27.183 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378636 2026-02-12T19:10:27.183 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 1-42949672973 2-64424509452 3-81604378636' 2026-02-12T19:10:27.183 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:27.183 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672973 2026-02-12T19:10:27.184 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:27.184 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T19:10:27.185 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672973 2026-02-12T19:10:27.185 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:27.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672973 2026-02-12T19:10:27.186 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.1 seq 42949672973 2026-02-12T19:10:27.187 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672973' 2026-02-12T19:10:27.187 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T19:10:27.400 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672971 -lt 42949672973 2026-02-12T19:10:27.401 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:10:28.401 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:10:28.402 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T19:10:28.619 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672973 -lt 42949672973 2026-02-12T19:10:28.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:28.620 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 2-64424509452 2026-02-12T19:10:28.620 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:28.621 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=2 2026-02-12T19:10:28.622 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509452 2026-02-12T19:10:28.622 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:28.624 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=64424509452 2026-02-12T19:10:28.624 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.2 seq 64424509452 2026-02-12T19:10:28.624 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.2 seq 64424509452' 2026-02-12T19:10:28.624 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-12T19:10:28.847 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509453 -lt 64424509452 2026-02-12T19:10:28.847 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:10:28.848 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 3-81604378636 2026-02-12T19:10:28.848 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:10:28.849 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=3 2026-02-12T19:10:28.849 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-81604378636 2026-02-12T19:10:28.849 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:10:28.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=81604378636 2026-02-12T19:10:28.851 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.3 seq 81604378636 2026-02-12T19:10:28.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.3 seq 81604378636' 2026-02-12T19:10:28.851 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-12T19:10:29.077 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 81604378636 -lt 81604378636 2026-02-12T19:10:29.078 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: get_num_pgs 2026-02-12T19:10:29.078 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:10:29.078 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:10:29.366 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1738: wait_for_peered: test 32 == 0 2026-02-12T19:10:29.366 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1742: wait_for_peered: true 2026-02-12T19:10:29.367 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: get_num_active_or_peered 2026-02-12T19:10:29.367 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-12T19:10:29.367 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-12T19:10:29.367 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-12T19:10:29.367 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-12T19:10:29.367 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-12T19:10:29.585 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1746: wait_for_peered: cur_peered=32 2026-02-12T19:10:29.586 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: get_num_pgs 2026-02-12T19:10:29.586 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:10:29.586 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:10:29.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: test 32 = 32 2026-02-12T19:10:29.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1747: wait_for_peered: break 2026-02-12T19:10:29.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1762: wait_for_peered: return 0 2026-02-12T19:10:29.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:289: TEST_0_osd: ceph osd ok-to-stop 0 2026-02-12T19:10:30.080 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[0],"num_ok_pgs":0,"num_not_ok_pgs":0} 2026-02-12T19:10:30.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:290: TEST_0_osd: ceph osd ok-to-stop 0 1 2026-02-12T19:10:30.292 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":true,"osds":[0,1],"num_ok_pgs":32,"num_not_ok_pgs":0,"ok_become_degraded":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:30.301 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:291: TEST_0_osd: ceph osd ok-to-stop 0 1 2 2026-02-12T19:10:30.449 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":false,"osds":[0,1,2],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:30.449 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-12T19:10:30.454 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:292: TEST_0_osd: ceph osd ok-to-stop 1 2 3 2026-02-12T19:10:30.605 INFO:tasks.workunit.client.0.trial010.stdout:{"ok_to_stop":false,"osds":[1,2,3],"num_ok_pgs":0,"num_not_ok_pgs":32,"bad_become_inactive":["1.0","1.1","1.2","1.3","1.4","1.5","1.6","1.7","1.8","1.9","1.a","1.b","1.c","1.d","1.e","1.f","1.10","1.11","1.12","1.13","1.14","1.15","1.16","1.17","1.18","1.19","1.1a","1.1b","1.1c","1.1d","1.1e","1.1f"]} 2026-02-12T19:10:30.606 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: unsafe to stop osd(s) at this time (32 PGs are or would become offline) 2026-02-12T19:10:30.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-12T19:10:30.609 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:10:30.609 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:10:30.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:10:30.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:10:30.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:10:30.720 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:10:30.720 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-12T19:10:30.720 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:10:30.721 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:10:30.721 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:10:30.721 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:10:30.721 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:10:30.721 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:10:30.721 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:10:30.721 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:10:30.726 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:10:30.727 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:10:30.728 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:10:30.728 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:10:30.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:10:30.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:10:30.729 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:10:30.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:30.731 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:10:30.731 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:10:30.731 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:30.732 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:10:30.733 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:10:30.733 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:10:30.748 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:10:30.748 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:30.748 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:30.748 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:10:30.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:10:30.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:10:30.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-12T19:10:30.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-12T19:10:30.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-12T19:10:30.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-12T19:10:30.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:10:30.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:10:30.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:10:30.751 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:10:30.751 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:10:30.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:10:30.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:10:30.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:10:30.752 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:10:30.753 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:10:30.753 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:10:30.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:10:30.754 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:10:30.755 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:10:30.755 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:10:30.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:30.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:10:30.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:10:30.757 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:30.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:10:30.759 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:10:30.759 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:10:30.760 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:10:30.760 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:30.760 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:30.760 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:10:30.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:10:30.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:10:30.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-12T19:10:30.763 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:10:30.763 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:30.763 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:30.763 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31636 2026-02-12T19:10:30.764 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:10:30.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:10:30.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:10:30.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-12T19:10:30.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_1_mon_checks td/ok-to-stop 2026-02-12T19:10:30.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:28: TEST_1_mon_checks: local dir=td/ok-to-stop 2026-02-12T19:10:30.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:30: TEST_1_mon_checks: CEPH_ARGS='--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150 ' 2026-02-12T19:10:30.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:32: TEST_1_mon_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-12T19:10:30.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:10:30.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:10:30.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:10:30.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:10:30.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-12T19:10:30.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-12T19:10:30.787 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:10:30.787 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:10:30.787 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:10:30.787 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:10:30.787 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:30.787 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:30.788 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:30.788 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-12T19:10:30.816 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:10:30.817 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:10:30.817 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:30.817 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:10:30.817 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:10:30.819 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:10:30.819 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:10:30.819 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:10:30.819 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:10:30.819 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:30.819 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:30.819 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:30.819 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:10:30.819 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:30.819 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get fsid 2026-02-12T19:10:30.875 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:10:30.875 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:30.876 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:10:30.876 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:10:30.877 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:10:30.877 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:10:30.877 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:10:30.877 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:10:30.877 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:30.877 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:30.877 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:30.878 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:10:30.878 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:30.878 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get mon_host 2026-02-12T19:10:30.932 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:34: TEST_1_mon_checks: ceph mon ok-to-stop dne 2026-02-12T19:10:31.034 INFO:tasks.workunit.client.0.trial010.stderr:invalid mon(s) specified: [dne] 2026-02-12T19:10:31.042 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:35: TEST_1_mon_checks: ceph mon ok-to-stop a 2026-02-12T19:10:31.142 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a] 2026-02-12T19:10:31.144 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:37: TEST_1_mon_checks: ceph mon ok-to-add-offline 2026-02-12T19:10:31.230 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-12T19:10:31.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:39: TEST_1_mon_checks: ceph mon ok-to-rm a 2026-02-12T19:10:31.319 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-12T19:10:31.322 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:40: TEST_1_mon_checks: ceph mon ok-to-rm dne 2026-02-12T19:10:31.429 INFO:tasks.workunit.client.0.trial010.stderr:mon.dne does not exist 2026-02-12T19:10:31.437 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-12T19:10:31.438 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:10:31.438 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:10:31.438 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:10:31.438 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:10:31.438 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:10:31.545 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:10:31.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-12T19:10:31.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:10:31.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:10:31.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:10:31.546 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:10:31.546 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:10:31.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:10:31.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:10:31.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:10:31.551 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:10:31.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:10:31.552 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:10:31.552 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:10:31.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:10:31.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:10:31.554 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:10:31.555 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:31.555 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:10:31.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:10:31.556 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:31.557 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:10:31.558 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:10:31.558 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:10:31.560 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:10:31.560 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:31.560 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:31.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:10:31.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:10:31.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:10:31.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-12T19:10:31.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-12T19:10:31.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-12T19:10:31.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-12T19:10:31.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:10:31.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:10:31.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:10:31.563 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:10:31.563 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:10:31.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:10:31.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:10:31.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:10:31.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:10:31.565 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:10:31.566 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:10:31.566 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:10:31.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:10:31.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:10:31.568 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:10:31.569 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:31.569 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:10:31.569 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:10:31.570 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:31.570 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:10:31.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:10:31.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:10:31.573 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:10:31.573 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:31.573 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:31.573 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:10:31.574 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:10:31.574 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:10:31.574 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-12T19:10:31.575 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:10:31.575 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:31.576 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:31.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31636 2026-02-12T19:10:31.579 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:10:31.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:10:31.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:10:31.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-12T19:10:31.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_2_mons_checks td/ok-to-stop 2026-02-12T19:10:31.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:44: TEST_2_mons_checks: local dir=td/ok-to-stop 2026-02-12T19:10:31.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:46: TEST_2_mons_checks: CEPH_ARGS='--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151 ' 2026-02-12T19:10:31.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:48: TEST_2_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-12T19:10:31.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:10:31.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:10:31.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:10:31.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:10:31.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-12T19:10:31.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-12T19:10:31.600 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:10:31.600 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:10:31.600 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:10:31.601 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:10:31.601 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:31.601 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:31.601 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:31.601 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-12T19:10:31.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:10:31.629 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:10:31.629 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:31.630 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:10:31.630 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:10:31.631 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:10:31.631 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:10:31.631 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:10:31.631 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:10:31.631 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:31.631 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:31.632 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:31.632 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:10:31.632 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:31.632 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get fsid 2026-02-12T19:10:31.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:10:31.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:31.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:10:31.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:10:31.689 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:10:31.689 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:10:31.689 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:10:31.689 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:10:31.690 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:31.690 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:31.690 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:31.690 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:10:31.690 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:31.691 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get mon_host 2026-02-12T19:10:31.745 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:49: TEST_2_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-12T19:10:31.745 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:10:31.745 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:10:31.745 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-12T19:10:31.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:10:31.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-12T19:10:31.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-12T19:10:31.764 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:10:31.764 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:10:31.764 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:10:31.765 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:10:31.765 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:31.765 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:31.765 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:31.765 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-12T19:10:31.792 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:10:31.793 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-12T19:10:31.793 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:31.793 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-12T19:10:31.793 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:10:31.793 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:10:31.794 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-12T19:10:31.794 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-12T19:10:31.794 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-12T19:10:31.794 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:31.794 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:31.794 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:31.794 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.b.asok 2026-02-12T19:10:31.795 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:31.795 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.b.asok config get fsid 2026-02-12T19:10:31.849 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-12T19:10:31.849 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:31.849 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-12T19:10:31.849 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:10:31.850 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:10:31.850 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-12T19:10:31.850 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-12T19:10:31.850 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-12T19:10:31.851 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:31.851 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:31.851 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:31.851 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.b.asok 2026-02-12T19:10:31.851 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:31.851 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.b.asok config get mon_host 2026-02-12T19:10:31.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:51: TEST_2_mons_checks: ceph mon ok-to-stop dne 2026-02-12T19:10:32.014 INFO:tasks.workunit.client.0.trial010.stderr:invalid mon(s) specified: [dne] 2026-02-12T19:10:32.022 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:52: TEST_2_mons_checks: ceph mon ok-to-stop a 2026-02-12T19:10:32.122 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (b) after stopping mons [a] 2026-02-12T19:10:32.125 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:53: TEST_2_mons_checks: ceph mon ok-to-stop b 2026-02-12T19:10:32.210 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b] 2026-02-12T19:10:32.214 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:54: TEST_2_mons_checks: ceph mon ok-to-stop a b 2026-02-12T19:10:32.302 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b] 2026-02-12T19:10:32.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:56: TEST_2_mons_checks: ceph mon ok-to-add-offline 2026-02-12T19:10:32.412 INFO:tasks.workunit.client.0.trial010.stderr:adding another mon that is not yet online will not break quorum 2026-02-12T19:10:32.420 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:58: TEST_2_mons_checks: ceph mon ok-to-rm a 2026-02-12T19:10:32.525 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.a 2026-02-12T19:10:32.535 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:59: TEST_2_mons_checks: ceph mon ok-to-rm b 2026-02-12T19:10:32.642 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.b 2026-02-12T19:10:32.651 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:60: TEST_2_mons_checks: ceph mon ok-to-rm dne 2026-02-12T19:10:32.770 INFO:tasks.workunit.client.0.trial010.stderr:mon.dne does not exist 2026-02-12T19:10:32.779 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-12T19:10:32.780 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:10:32.780 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:10:32.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:10:32.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:10:32.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:10:32.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:10:32.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-12T19:10:32.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:10:32.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:10:32.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:10:32.887 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:10:32.887 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:10:32.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:10:32.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:10:32.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:10:32.893 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:10:32.893 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:10:32.894 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:10:32.895 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:10:32.896 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:10:32.896 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:10:32.896 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:10:32.897 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:32.897 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:10:32.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:10:32.898 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:32.899 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:10:32.900 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:10:32.900 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:10:32.903 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:10:32.903 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:32.903 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:32.904 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:10:32.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:10:32.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:10:32.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-12T19:10:32.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-12T19:10:32.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-12T19:10:32.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-12T19:10:32.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:10:32.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:10:32.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:10:32.906 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:10:32.906 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:10:32.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:10:32.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:10:32.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:10:32.908 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:10:32.908 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:10:32.909 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:10:32.910 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:10:32.911 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:10:32.911 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:10:32.911 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:10:32.912 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:32.912 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:10:32.913 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:10:32.913 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:10:32.914 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:10:32.915 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:10:32.915 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:10:32.916 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:10:32.916 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:32.916 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:32.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:10:32.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:10:32.917 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:10:32.918 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-12T19:10:32.919 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:10:32.919 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:32.919 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:32.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31636 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_3_mons_checks td/ok-to-stop 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:64: TEST_3_mons_checks: local dir=td/ok-to-stop 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:66: TEST_3_mons_checks: CEPH_ARGS='--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152 ' 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:68: TEST_3_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-12T19:10:32.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-12T19:10:32.944 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:10:32.944 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:10:32.944 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:10:32.944 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:10:32.944 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:32.944 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:32.945 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:32.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-12T19:10:32.982 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:10:32.983 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:10:32.983 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:32.983 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:10:32.983 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:10:32.985 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:10:32.985 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:10:32.985 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:10:32.985 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:10:32.985 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:32.985 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:32.985 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:32.985 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:10:32.985 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:32.985 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get fsid 2026-02-12T19:10:33.042 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:10:33.042 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:33.042 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:10:33.042 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:10:33.042 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:10:33.044 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:10:33.044 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:10:33.044 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:10:33.044 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:33.044 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:33.044 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:33.044 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:10:33.044 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:33.044 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get mon_host 2026-02-12T19:10:33.100 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:69: TEST_3_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-12T19:10:33.100 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:10:33.100 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:10:33.100 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-12T19:10:33.100 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:10:33.100 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-12T19:10:33.100 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-12T19:10:33.120 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:10:33.120 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:10:33.120 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:10:33.120 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:10:33.120 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:33.120 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:33.121 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:33.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-12T19:10:33.149 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:10:33.150 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-12T19:10:33.150 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:33.150 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-12T19:10:33.151 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:10:33.152 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:10:33.152 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-12T19:10:33.152 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-12T19:10:33.152 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-12T19:10:33.152 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:33.152 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:33.152 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:33.153 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.b.asok 2026-02-12T19:10:33.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:33.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.b.asok config get fsid 2026-02-12T19:10:33.209 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-12T19:10:33.209 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:33.210 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-12T19:10:33.210 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:10:33.211 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:10:33.211 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-12T19:10:33.211 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-12T19:10:33.211 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-12T19:10:33.211 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:33.211 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:33.211 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:33.211 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.b.asok 2026-02-12T19:10:33.212 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:33.212 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.b.asok config get mon_host 2026-02-12T19:10:33.268 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:70: TEST_3_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-12T19:10:33.268 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:10:33.268 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:10:33.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-12T19:10:33.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:10:33.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-12T19:10:33.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-12T19:10:33.289 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:10:33.289 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:10:33.289 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:10:33.289 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:10:33.289 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:33.289 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:33.290 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:10:33.290 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-12T19:10:33.319 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:10:33.320 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-12T19:10:33.320 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:33.320 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-12T19:10:33.321 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:10:33.322 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:10:33.322 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-12T19:10:33.322 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-12T19:10:33.322 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-12T19:10:33.322 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:33.322 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:33.322 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:33.322 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.c.asok 2026-02-12T19:10:33.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:33.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.c.asok config get fsid 2026-02-12T19:10:33.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-12T19:10:33.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:10:33.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-12T19:10:33.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:10:33.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:10:33.381 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-12T19:10:33.381 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-12T19:10:33.382 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-12T19:10:33.382 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:10:33.382 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:10:33.382 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:10:33.382 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.c.asok 2026-02-12T19:10:33.382 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:10:33.382 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.c.asok config get mon_host 2026-02-12T19:10:33.438 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:71: TEST_3_mons_checks: wait_for_quorum 60 3 2026-02-12T19:10:33.438 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-12T19:10:33.438 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-12T19:10:33.439 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-12T19:10:33.439 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-12T19:10:33.439 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-12T19:10:33.439 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-12T19:10:33.439 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770923493 2026-02-12T19:10:33.440 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:33.441 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923433 -lt 1770923493 ]] 2026-02-12T19:10:33.441 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:10:33.441 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:33.754 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:33.754 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:33.755 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:33.755 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:10:33.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-12T19:10:33.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-12T19:10:33.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-12T19:10:33.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-12T19:10:33.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-12T19:10:33.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:73: TEST_3_mons_checks: ceph mon ok-to-stop dne 2026-02-12T19:10:33.872 INFO:tasks.workunit.client.0.trial010.stderr:invalid mon(s) specified: [dne] 2026-02-12T19:10:33.883 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:74: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-12T19:10:33.988 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (b,c) after stopping [a] 2026-02-12T19:10:33.996 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:75: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-12T19:10:34.114 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-12T19:10:34.125 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:76: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-12T19:10:34.231 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (a,b) after stopping [c] 2026-02-12T19:10:34.242 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:77: TEST_3_mons_checks: ceph mon ok-to-stop a b 2026-02-12T19:10:34.332 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a,b] 2026-02-12T19:10:34.334 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:78: TEST_3_mons_checks: ceph mon ok-to-stop b c 2026-02-12T19:10:34.425 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [b,c] 2026-02-12T19:10:34.427 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:79: TEST_3_mons_checks: ceph mon ok-to-stop a b c 2026-02-12T19:10:34.511 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available () after stopping mons [a,b,c] 2026-02-12T19:10:34.515 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:81: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-12T19:10:34.617 INFO:tasks.workunit.client.0.trial010.stderr:adding another mon that is not yet online will not break quorum 2026-02-12T19:10:34.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:83: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-12T19:10:34.734 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.a 2026-02-12T19:10:34.744 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:84: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-12T19:10:34.850 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.b 2026-02-12T19:10:34.858 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:85: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-12T19:10:37.996 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.c 2026-02-12T19:10:38.004 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:87: TEST_3_mons_checks: kill_daemons td/ok-to-stop KILL mon.b 2026-02-12T19:10:38.005 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:10:38.005 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:10:38.005 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:10:38.005 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:10:38.005 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:10:38.112 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:10:38.113 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:88: TEST_3_mons_checks: wait_for_quorum 60 2 2026-02-12T19:10:38.113 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-12T19:10:38.113 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=2 2026-02-12T19:10:38.113 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-12T19:10:38.113 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 2 ]] 2026-02-12T19:10:38.113 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-12T19:10:38.113 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-12T19:10:38.114 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770923498 2026-02-12T19:10:38.114 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:38.115 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923438 -lt 1770923498 ]] 2026-02-12T19:10:38.115 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:38.115 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:38.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:38.228 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:38.229 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:38.229 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:38.238 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:38.238 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:38.238 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:38.239 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923438 -lt 1770923498 ]] 2026-02-12T19:10:38.239 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:38.240 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:38.355 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:38.355 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:38.356 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:38.356 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:38.365 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:38.365 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:38.366 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:38.367 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923438 -lt 1770923498 ]] 2026-02-12T19:10:38.367 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:38.367 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:38.485 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:38.485 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:38.485 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:38.486 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:38.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:38.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:38.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:38.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923438 -lt 1770923498 ]] 2026-02-12T19:10:38.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:38.496 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:41.617 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:41.617 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:41.618 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:41.618 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:41.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:41.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:41.627 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:41.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923441 -lt 1770923498 ]] 2026-02-12T19:10:41.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:41.629 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:41.739 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:41.740 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:41.740 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:41.740 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:41.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:41.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:41.750 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:41.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923441 -lt 1770923498 ]] 2026-02-12T19:10:41.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:41.752 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:44.869 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:44.870 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:44.870 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:44.871 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:44.880 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:44.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:44.881 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:44.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923444 -lt 1770923498 ]] 2026-02-12T19:10:44.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:44.883 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:44.997 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:44.997 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:44.997 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:44.998 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.007 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:45.007 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:45.007 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:45.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923445 -lt 1770923498 ]] 2026-02-12T19:10:45.009 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:45.009 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:45.125 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:45.126 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.126 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.127 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:45.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:45.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:45.137 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:45.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923445 -lt 1770923498 ]] 2026-02-12T19:10:45.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:45.139 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:45.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:45.252 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.253 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:45.253 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.262 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:45.263 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:45.263 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:45.264 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923445 -lt 1770923498 ]] 2026-02-12T19:10:45.264 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:45.264 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:45.378 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:45.378 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.379 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:45.379 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.389 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:45.389 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:45.389 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:45.390 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923445 -lt 1770923498 ]] 2026-02-12T19:10:45.391 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:45.391 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:45.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:45.503 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.504 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.504 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:45.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:45.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:45.515 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:45.516 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923445 -lt 1770923498 ]] 2026-02-12T19:10:45.516 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:45.516 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:45.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:45.629 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.630 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:45.630 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.641 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:45.641 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:45.641 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:45.642 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923445 -lt 1770923498 ]] 2026-02-12T19:10:45.642 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:45.643 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:45.754 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:45.754 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:45.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:45.765 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:10:45.765 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:10:45.765 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:10:45.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923445 -lt 1770923498 ]] 2026-02-12T19:10:45.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 2' 2026-02-12T19:10:45.767 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:10:53.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:10:53.293 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":8,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:53.293 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 2' 2026-02-12T19:10:53.294 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:10:32.936110Z","created":"2026-02-12T19:10:32.936110Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936107+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936108+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:10:32:936109+0000","crush_location":[]}]}}' 2026-02-12T19:10:53.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-12T19:10:53.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-12T19:10:53.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-12T19:10:53.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-12T19:10:53.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-12T19:10:53.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:90: TEST_3_mons_checks: ceph mon ok-to-stop a 2026-02-12T19:10:53.390 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (c) after stopping mons [a] 2026-02-12T19:10:53.392 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:91: TEST_3_mons_checks: ceph mon ok-to-stop b 2026-02-12T19:10:53.497 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (a,c) after stopping [b] 2026-02-12T19:10:53.507 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:92: TEST_3_mons_checks: ceph mon ok-to-stop c 2026-02-12T19:10:56.615 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (a) after stopping mons [c] 2026-02-12T19:10:56.618 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:94: TEST_3_mons_checks: ceph mon ok-to-add-offline 2026-02-12T19:10:59.707 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-12T19:10:59.709 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:96: TEST_3_mons_checks: ceph mon ok-to-rm a 2026-02-12T19:10:59.796 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: removing mon.a would break quorum 2026-02-12T19:10:59.799 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:97: TEST_3_mons_checks: ceph mon ok-to-rm b 2026-02-12T19:10:59.905 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.b 2026-02-12T19:10:59.913 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:98: TEST_3_mons_checks: ceph mon ok-to-rm c 2026-02-12T19:11:00.012 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-12T19:11:00.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-12T19:11:00.016 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:11:00.016 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:11:00.016 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:11:00.017 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:11:00.017 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:11:00.125 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:11:00.125 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-12T19:11:00.125 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:11:00.125 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:11:00.125 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:11:00.125 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:11:00.125 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:11:00.126 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:11:00.126 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:11:00.126 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:11:00.131 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:11:00.132 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:11:00.132 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:11:00.133 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:11:00.134 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:11:00.134 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:11:00.135 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:11:00.136 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:11:00.136 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:11:00.136 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:11:00.137 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:11:00.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:11:00.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:11:00.139 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:11:00.143 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:11:00.143 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.143 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.143 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:11:00.145 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:11:00.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:11:00.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-12T19:11:00.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-12T19:11:00.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-12T19:11:00.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-12T19:11:00.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:11:00.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:11:00.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:11:00.146 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:11:00.146 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:11:00.147 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:11:00.147 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:11:00.147 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:11:00.148 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:11:00.148 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:11:00.149 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:11:00.149 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:11:00.150 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:11:00.150 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:11:00.151 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:11:00.152 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:11:00.152 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:11:00.152 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:11:00.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:11:00.153 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:11:00.154 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:11:00.154 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:11:00.156 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:11:00.156 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.156 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.156 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:11:00.157 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:11:00.157 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:11:00.157 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-12T19:11:00.158 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:11:00.159 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.159 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.159 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31636 2026-02-12T19:11:00.160 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:11:00.162 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:11:00.162 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:11:00.162 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-12T19:11:00.162 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_4_mons_checks td/ok-to-stop 2026-02-12T19:11:00.162 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:102: TEST_4_mons_checks: local dir=td/ok-to-stop 2026-02-12T19:11:00.162 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:104: TEST_4_mons_checks: CEPH_ARGS='--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153 ' 2026-02-12T19:11:00.162 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:106: TEST_4_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-12T19:11:00.162 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:11:00.163 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:11:00.163 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:11:00.163 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:11:00.163 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-12T19:11:00.163 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-12T19:11:00.181 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:11:00.182 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:11:00.182 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:11:00.182 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:11:00.182 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.182 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.182 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:11:00.183 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-12T19:11:00.209 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:11:00.210 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:11:00.211 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:00.211 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:11:00.211 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:11:00.212 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:11:00.212 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:11:00.212 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:11:00.212 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:11:00.212 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:00.212 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.212 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.213 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:11:00.213 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:00.213 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get fsid 2026-02-12T19:11:00.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:11:00.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:00.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:11:00.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:11:00.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:11:00.269 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:11:00.269 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:11:00.269 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:11:00.270 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:00.270 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.270 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.270 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:11:00.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:00.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get mon_host 2026-02-12T19:11:00.323 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:107: TEST_4_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-12T19:11:00.324 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:11:00.324 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:11:00.324 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-12T19:11:00.324 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:11:00.324 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-12T19:11:00.324 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-12T19:11:00.343 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:11:00.343 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:11:00.343 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:11:00.344 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:11:00.344 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.344 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.344 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:11:00.345 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-12T19:11:00.370 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:11:00.371 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-12T19:11:00.371 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:00.371 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-12T19:11:00.371 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:11:00.373 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:11:00.373 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-12T19:11:00.373 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-12T19:11:00.373 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-12T19:11:00.373 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:00.373 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.373 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.373 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.b.asok 2026-02-12T19:11:00.373 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:00.373 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.b.asok config get fsid 2026-02-12T19:11:00.428 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-12T19:11:00.428 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:00.428 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-12T19:11:00.428 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:11:00.429 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:11:00.430 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-12T19:11:00.430 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-12T19:11:00.430 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-12T19:11:00.430 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:00.430 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.430 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.430 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.b.asok 2026-02-12T19:11:00.430 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:00.430 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.b.asok config get mon_host 2026-02-12T19:11:00.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:108: TEST_4_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-12T19:11:00.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:11:00.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:11:00.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-12T19:11:00.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:11:00.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-12T19:11:00.485 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-12T19:11:00.504 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:11:00.504 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:11:00.504 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:11:00.504 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:11:00.504 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.504 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:11:00.505 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-12T19:11:00.533 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:11:00.534 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-12T19:11:00.535 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:00.535 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-12T19:11:00.535 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:11:00.535 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:11:00.536 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-12T19:11:00.536 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-12T19:11:00.536 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-12T19:11:00.536 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:00.536 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.537 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.537 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.c.asok 2026-02-12T19:11:00.537 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:00.537 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.c.asok config get fsid 2026-02-12T19:11:00.592 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-12T19:11:00.592 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:00.592 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-12T19:11:00.592 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:11:00.593 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:11:00.593 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-12T19:11:00.593 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-12T19:11:00.594 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-12T19:11:00.594 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:00.594 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.594 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.594 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.c.asok 2026-02-12T19:11:00.594 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:00.594 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.c.asok config get mon_host 2026-02-12T19:11:00.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:109: TEST_4_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-12T19:11:00.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:11:00.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:11:00.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-12T19:11:00.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:11:00.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-12T19:11:00.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-12T19:11:00.668 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:11:00.668 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:11:00.668 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:11:00.668 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:11:00.668 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.669 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.669 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:11:00.669 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-12T19:11:00.695 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:11:00.695 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-12T19:11:00.696 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:00.696 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-12T19:11:00.696 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:11:00.697 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:11:00.697 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-12T19:11:00.697 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-12T19:11:00.697 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-12T19:11:00.698 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:00.698 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.698 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.698 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.d.asok 2026-02-12T19:11:00.698 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:00.698 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.d.asok config get fsid 2026-02-12T19:11:00.753 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-12T19:11:00.753 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:00.753 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-12T19:11:00.753 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:11:00.753 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:11:00.755 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-12T19:11:00.755 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-12T19:11:00.755 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-12T19:11:00.755 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:00.755 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:00.755 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:00.755 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.d.asok 2026-02-12T19:11:00.755 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:00.755 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.d.asok config get mon_host 2026-02-12T19:11:00.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:110: TEST_4_mons_checks: wait_for_quorum 60 4 2026-02-12T19:11:00.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-12T19:11:00.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-12T19:11:00.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-12T19:11:00.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-12T19:11:00.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-12T19:11:00.809 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-12T19:11:00.810 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770923520 2026-02-12T19:11:00.810 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:00.811 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923460 -lt 1770923520 ]] 2026-02-12T19:11:00.811 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-12T19:11:00.811 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:01.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:01.123 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:01.124 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:01.124 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-12T19:11:01.134 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-12T19:11:01.134 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-12T19:11:01.134 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-12T19:11:01.134 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-12T19:11:01.134 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-12T19:11:01.135 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:112: TEST_4_mons_checks: ceph mon ok-to-stop dne 2026-02-12T19:11:01.238 INFO:tasks.workunit.client.0.trial010.stderr:invalid mon(s) specified: [dne] 2026-02-12T19:11:01.248 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:113: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-12T19:11:01.352 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-12T19:11:01.360 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:114: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-12T19:11:01.479 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (a,c,d) after stopping [b] 2026-02-12T19:11:01.486 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:115: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-12T19:11:01.602 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (a,b,d) after stopping [c] 2026-02-12T19:11:01.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:116: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-12T19:11:04.746 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (a,b,c) after stopping [d] 2026-02-12T19:11:04.754 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:117: TEST_4_mons_checks: ceph mon ok-to-stop a b 2026-02-12T19:11:07.849 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [a,b] 2026-02-12T19:11:07.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:118: TEST_4_mons_checks: ceph mon ok-to-stop c d 2026-02-12T19:11:07.940 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (a,b) after stopping mons [c,d] 2026-02-12T19:11:07.944 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:120: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-12T19:11:08.048 INFO:tasks.workunit.client.0.trial010.stderr:adding another mon that is not yet online will not break quorum 2026-02-12T19:11:08.056 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:122: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-12T19:11:11.194 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.a 2026-02-12T19:11:11.201 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:123: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-12T19:11:11.314 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.b 2026-02-12T19:11:11.322 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:124: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-12T19:11:11.439 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.c 2026-02-12T19:11:11.446 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:126: TEST_4_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-12T19:11:11.447 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:11:11.448 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:11:11.448 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:11:11.448 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:11:11.448 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:11:11.555 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:11:11.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:127: TEST_4_mons_checks: wait_for_quorum 60 3 2026-02-12T19:11:11.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-12T19:11:11.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-12T19:11:11.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-12T19:11:11.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-12T19:11:11.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-12T19:11:11.556 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-12T19:11:11.557 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770923531 2026-02-12T19:11:11.558 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:11.558 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923471 -lt 1770923531 ]] 2026-02-12T19:11:11.558 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:11.558 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:14.677 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:14.677 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:14.678 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:14.678 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:14.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:14.687 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:14.687 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:14.687 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923474 -lt 1770923531 ]] 2026-02-12T19:11:14.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:14.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:14.799 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:14.799 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:14.800 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:14.800 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:14.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:14.810 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:14.810 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:14.811 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923474 -lt 1770923531 ]] 2026-02-12T19:11:14.811 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:14.811 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:14.926 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:14.926 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:14.927 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:14.927 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:14.937 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:14.937 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:14.938 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:14.939 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923474 -lt 1770923531 ]] 2026-02-12T19:11:14.939 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:14.939 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:15.050 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:15.051 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.051 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.052 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:15.062 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:15.062 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:15.062 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:15.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923475 -lt 1770923531 ]] 2026-02-12T19:11:15.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:15.064 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:15.180 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:15.181 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.182 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.182 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:15.191 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:15.191 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:15.192 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:15.192 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923475 -lt 1770923531 ]] 2026-02-12T19:11:15.193 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:15.193 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:15.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:15.307 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.307 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:15.308 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.317 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:15.317 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:15.318 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:15.319 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923475 -lt 1770923531 ]] 2026-02-12T19:11:15.319 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:15.319 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:15.431 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:15.431 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.432 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:15.432 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.442 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:15.442 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:15.442 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:15.443 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923475 -lt 1770923531 ]] 2026-02-12T19:11:15.443 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:15.444 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:15.558 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:15.558 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.559 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:15.559 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.569 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:15.569 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:15.570 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:15.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923475 -lt 1770923531 ]] 2026-02-12T19:11:15.571 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:15.572 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:15.682 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:15.682 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.683 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:15.683 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.691 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:15.692 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:15.692 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:15.693 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923475 -lt 1770923531 ]] 2026-02-12T19:11:15.693 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:15.694 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:15.808 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:15.808 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.809 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:15.810 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.818 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:15.818 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:15.819 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:15.819 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923475 -lt 1770923531 ]] 2026-02-12T19:11:15.820 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:15.820 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:15.934 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:15.935 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.936 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:15.936 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:15.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:15.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:15.946 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:15.947 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923475 -lt 1770923531 ]] 2026-02-12T19:11:15.947 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:15.947 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:16.062 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:16.062 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:16.063 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:16.063 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:16.073 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:16.073 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:16.074 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:16.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923476 -lt 1770923531 ]] 2026-02-12T19:11:16.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:16.075 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:16.190 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:16.191 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:16.191 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:16.192 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:16.202 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:16.202 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:16.202 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:16.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923476 -lt 1770923531 ]] 2026-02-12T19:11:16.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:16.204 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:16.318 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:16.319 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:16.320 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:16.320 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:16.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:16.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:16.330 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:16.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923476 -lt 1770923531 ]] 2026-02-12T19:11:16.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:16.332 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:16.442 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:16.443 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:16.443 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:16.443 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:16.453 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:16.453 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:16.454 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:16.455 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923476 -lt 1770923531 ]] 2026-02-12T19:11:16.455 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:16.455 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:19.570 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:19.571 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:19.571 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:19.571 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:19.581 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:19.581 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:19.582 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:19.583 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923479 -lt 1770923531 ]] 2026-02-12T19:11:19.583 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:19.583 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:19.692 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:19.692 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:19.693 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:19.693 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:19.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:19.703 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:19.703 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:19.704 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923479 -lt 1770923531 ]] 2026-02-12T19:11:19.704 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:19.704 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:19.819 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:19.820 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:19.820 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:19.821 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:19.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:19.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:19.831 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:19.832 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923479 -lt 1770923531 ]] 2026-02-12T19:11:19.832 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:19.832 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:19.948 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:19.948 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:19.949 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:19.949 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:19.959 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:19.959 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:19.959 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:19.960 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923479 -lt 1770923531 ]] 2026-02-12T19:11:19.960 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:19.961 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:20.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:20.075 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:20.076 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":4,"quorum":[0,1,2,3],"quorum_names":["a","b","c","d"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:20.076 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:20.086 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:20.086 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:20.086 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:20.087 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923480 -lt 1770923531 ]] 2026-02-12T19:11:20.087 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:11:20.088 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:26.810 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:26.810 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":6,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:26.811 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:11:26.811 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":6,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:00.173047Z","created":"2026-02-12T19:11:00.173047Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173041+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173045+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:00:173046+0000","crush_location":[]}]}}' 2026-02-12T19:11:26.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-12T19:11:26.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-12T19:11:26.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-12T19:11:26.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-12T19:11:26.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-12T19:11:26.823 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:129: TEST_4_mons_checks: ceph mon ok-to-stop a 2026-02-12T19:11:26.927 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-12T19:11:26.938 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:130: TEST_4_mons_checks: ceph mon ok-to-stop b 2026-02-12T19:11:27.024 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-12T19:11:27.026 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:131: TEST_4_mons_checks: ceph mon ok-to-stop c 2026-02-12T19:11:27.112 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-12T19:11:27.115 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:132: TEST_4_mons_checks: ceph mon ok-to-stop d 2026-02-12T19:11:27.202 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-12T19:11:27.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:134: TEST_4_mons_checks: ceph mon ok-to-add-offline 2026-02-12T19:11:27.321 INFO:tasks.workunit.client.0.trial010.stderr:adding another mon that is not yet online will not break quorum 2026-02-12T19:11:27.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:136: TEST_4_mons_checks: ceph mon ok-to-rm a 2026-02-12T19:11:30.438 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.a 2026-02-12T19:11:30.446 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:137: TEST_4_mons_checks: ceph mon ok-to-rm b 2026-02-12T19:11:30.550 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.b 2026-02-12T19:11:30.558 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:138: TEST_4_mons_checks: ceph mon ok-to-rm c 2026-02-12T19:11:30.675 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.c 2026-02-12T19:11:30.683 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:139: TEST_4_mons_checks: ceph mon ok-to-rm d 2026-02-12T19:11:30.798 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.d 2026-02-12T19:11:30.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-12T19:11:30.810 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:11:30.810 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:11:30.810 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:11:30.810 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:11:30.810 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:11:30.918 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:11:30.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-12T19:11:30.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:11:30.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:11:30.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:11:30.919 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:11:30.919 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:11:30.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:11:30.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:11:30.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:11:30.924 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:11:30.925 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:11:30.925 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:11:30.926 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:11:30.927 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:11:30.927 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:11:30.928 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:11:30.928 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:11:30.928 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:11:30.929 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:11:30.929 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:11:30.930 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:11:30.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:11:30.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:11:30.938 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:11:30.938 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:30.938 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:30.938 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:11:30.939 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:19: run: for func in $funcs 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:20: run: setup td/ok-to-stop 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ok-to-stop 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ok-to-stop 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:11:30.940 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:11:30.943 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:11:30.943 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:11:30.944 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:11:30.944 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:11:30.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:11:30.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:11:30.945 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:11:30.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:11:30.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:11:30.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:11:30.947 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:11:30.948 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:11:30.949 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:11:30.949 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:11:30.950 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:11:30.950 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:30.950 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:30.950 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:11:30.952 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:11:30.952 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:11:30.952 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ok-to-stop 2026-02-12T19:11:30.953 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:11:30.953 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:30.953 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:30.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.31636 2026-02-12T19:11:30.955 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:11:30.955 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:11:30.955 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:11:30.955 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ok-to-stop 1' TERM HUP INT 2026-02-12T19:11:30.955 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:21: run: TEST_5_mons_checks td/ok-to-stop 2026-02-12T19:11:30.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:143: TEST_5_mons_checks: local dir=td/ok-to-stop 2026-02-12T19:11:30.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:145: TEST_5_mons_checks: CEPH_ARGS='--fsid=9dfe8ccd-165d-45ee-a6bb-cf436306c0de --auth-supported=none --mon-host=127.0.0.1:7150,127.0.0.1:7151,127.0.0.1:7152,127.0.0.1:7153,127.0.0.1:7154 ' 2026-02-12T19:11:30.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:147: TEST_5_mons_checks: run_mon td/ok-to-stop a --public-addr=127.0.0.1:7150 2026-02-12T19:11:30.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:11:30.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:11:30.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:11:30.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:11:30.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/a 2026-02-12T19:11:30.956 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ok-to-stop/a --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7150 2026-02-12T19:11:30.977 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:11:30.977 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:11:30.977 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:11:30.977 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:11:30.977 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:30.977 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:30.978 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:11:30.978 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/a '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7150 2026-02-12T19:11:31.005 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:11:31.005 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:11:31.006 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:31.006 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:11:31.006 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:11:31.006 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:11:31.006 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:11:31.006 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:11:31.006 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:11:31.007 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:31.007 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.007 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.007 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:11:31.008 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:31.008 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get fsid 2026-02-12T19:11:31.064 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:11:31.065 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:31.065 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:11:31.065 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:11:31.065 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:11:31.065 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:11:31.065 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:11:31.065 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:11:31.065 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:31.065 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.066 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.066 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.a.asok 2026-02-12T19:11:31.066 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:31.066 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.a.asok config get mon_host 2026-02-12T19:11:31.120 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:148: TEST_5_mons_checks: run_mon td/ok-to-stop b --public-addr=127.0.0.1:7151 2026-02-12T19:11:31.120 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:11:31.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:11:31.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=b 2026-02-12T19:11:31.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:11:31.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/b 2026-02-12T19:11:31.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id b --mkfs --mon-data=td/ok-to-stop/b --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7151 2026-02-12T19:11:31.140 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:11:31.141 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:11:31.141 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:11:31.141 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:11:31.141 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.141 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.141 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:11:31.142 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id b --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/b '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7151 2026-02-12T19:11:31.167 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:11:31.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b fsid 2026-02-12T19:11:31.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:31.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-12T19:11:31.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:11:31.170 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:11:31.170 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-12T19:11:31.170 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-12T19:11:31.170 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-12T19:11:31.170 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:31.170 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.170 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.170 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.b.asok 2026-02-12T19:11:31.171 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:31.171 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.b.asok config get fsid 2026-02-12T19:11:31.226 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon b mon_host 2026-02-12T19:11:31.226 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:31.226 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=b 2026-02-12T19:11:31.226 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:11:31.226 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:11:31.228 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.b 2026-02-12T19:11:31.228 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.b 2026-02-12T19:11:31.228 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.b ']' 2026-02-12T19:11:31.228 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:31.228 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.228 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.228 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.b.asok 2026-02-12T19:11:31.228 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:31.228 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.b.asok config get mon_host 2026-02-12T19:11:31.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:149: TEST_5_mons_checks: run_mon td/ok-to-stop c --public-addr=127.0.0.1:7152 2026-02-12T19:11:31.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:11:31.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:11:31.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=c 2026-02-12T19:11:31.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:11:31.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/c 2026-02-12T19:11:31.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id c --mkfs --mon-data=td/ok-to-stop/c --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7152 2026-02-12T19:11:31.304 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:11:31.305 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:11:31.305 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:11:31.305 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:11:31.305 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.305 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.305 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:11:31.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id c --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/c '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7152 2026-02-12T19:11:31.328 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:11:31.330 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c fsid 2026-02-12T19:11:31.331 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:31.331 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-12T19:11:31.331 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:11:31.331 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:11:31.331 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-12T19:11:31.331 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-12T19:11:31.331 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-12T19:11:31.331 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:31.331 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.331 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.332 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.c.asok 2026-02-12T19:11:31.332 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:31.333 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.c.asok config get fsid 2026-02-12T19:11:31.388 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon c mon_host 2026-02-12T19:11:31.388 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:31.389 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=c 2026-02-12T19:11:31.389 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:11:31.389 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:11:31.389 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.c 2026-02-12T19:11:31.389 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.c 2026-02-12T19:11:31.389 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.c ']' 2026-02-12T19:11:31.389 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:31.389 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.389 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.390 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.c.asok 2026-02-12T19:11:31.390 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:31.390 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.c.asok config get mon_host 2026-02-12T19:11:31.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:150: TEST_5_mons_checks: run_mon td/ok-to-stop d --public-addr=127.0.0.1:7153 2026-02-12T19:11:31.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:11:31.445 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:11:31.445 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=d 2026-02-12T19:11:31.445 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:11:31.445 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/d 2026-02-12T19:11:31.445 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id d --mkfs --mon-data=td/ok-to-stop/d --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7153 2026-02-12T19:11:31.463 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:11:31.463 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:11:31.463 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:11:31.463 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:11:31.463 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.463 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.463 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:11:31.464 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id d --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/d '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7153 2026-02-12T19:11:31.492 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:11:31.493 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d fsid 2026-02-12T19:11:31.493 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:31.493 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-12T19:11:31.493 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:11:31.493 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:11:31.494 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-12T19:11:31.494 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-12T19:11:31.494 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-12T19:11:31.494 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:31.494 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.494 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.494 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.d.asok 2026-02-12T19:11:31.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:31.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.d.asok config get fsid 2026-02-12T19:11:31.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon d mon_host 2026-02-12T19:11:31.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:31.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=d 2026-02-12T19:11:31.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:11:31.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:11:31.551 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.d 2026-02-12T19:11:31.551 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.d 2026-02-12T19:11:31.551 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.d ']' 2026-02-12T19:11:31.552 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:31.552 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.552 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.552 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.d.asok 2026-02-12T19:11:31.552 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:31.552 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.d.asok config get mon_host 2026-02-12T19:11:31.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:151: TEST_5_mons_checks: run_mon td/ok-to-stop e --public-addr=127.0.0.1:7154 2026-02-12T19:11:31.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ok-to-stop 2026-02-12T19:11:31.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:11:31.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=e 2026-02-12T19:11:31.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:11:31.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ok-to-stop/e 2026-02-12T19:11:31.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id e --mkfs --mon-data=td/ok-to-stop/e --run-dir=td/ok-to-stop --public-addr=127.0.0.1:7154 2026-02-12T19:11:31.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:11:31.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:11:31.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:11:31.629 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:11:31.629 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.629 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.629 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.31636/$cluster-$name.asok' 2026-02-12T19:11:31.630 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id e --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ok-to-stop/e '--log-file=td/ok-to-stop/$name.log' '--admin-socket=/tmp/ceph-asok.31636/$cluster-$name.asok' --mon-cluster-log-file=td/ok-to-stop/log --run-dir=td/ok-to-stop '--pid-file=td/ok-to-stop/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --public-addr=127.0.0.1:7154 2026-02-12T19:11:31.656 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:11:31.657 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e fsid 2026-02-12T19:11:31.657 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:31.657 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-12T19:11:31.658 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:11:31.659 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:11:31.659 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-12T19:11:31.659 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-12T19:11:31.659 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-12T19:11:31.659 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:31.659 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.659 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.659 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.e.asok 2026-02-12T19:11:31.660 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:31.660 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.e.asok config get fsid 2026-02-12T19:11:31.716 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon e mon_host 2026-02-12T19:11:31.716 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:11:31.717 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=e 2026-02-12T19:11:31.717 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:11:31.717 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:11:31.718 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.e 2026-02-12T19:11:31.718 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.e 2026-02-12T19:11:31.718 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.e ']' 2026-02-12T19:11:31.718 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:11:31.718 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:11:31.718 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:11:31.719 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.31636/ceph-mon.e.asok 2026-02-12T19:11:31.719 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:11:31.719 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.31636/ceph-mon.e.asok config get mon_host 2026-02-12T19:11:31.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:152: TEST_5_mons_checks: wait_for_quorum 60 5 2026-02-12T19:11:31.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-12T19:11:31.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=5 2026-02-12T19:11:31.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-12T19:11:31.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 5 ]] 2026-02-12T19:11:31.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-12T19:11:31.774 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-12T19:11:31.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770923551 2026-02-12T19:11:31.775 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:31.776 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923491 -lt 1770923551 ]] 2026-02-12T19:11:31.776 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 5' 2026-02-12T19:11:31.777 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:37.896 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:37.896 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:37.896 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 5' 2026-02-12T19:11:37.896 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:37.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-12T19:11:37.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-12T19:11:37.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-12T19:11:37.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-12T19:11:37.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-12T19:11:37.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:154: TEST_5_mons_checks: ceph mon ok-to-stop dne 2026-02-12T19:11:38.011 INFO:tasks.workunit.client.0.trial010.stderr:invalid mon(s) specified: [dne] 2026-02-12T19:11:38.020 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:155: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-12T19:11:38.124 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-12T19:11:38.132 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:156: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-12T19:11:38.238 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (a,c,d,e) after stopping [b] 2026-02-12T19:11:38.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:157: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-12T19:11:38.366 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (a,b,d,e) after stopping [c] 2026-02-12T19:11:38.373 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:158: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-12T19:11:38.477 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (a,b,c,e) after stopping [d] 2026-02-12T19:11:38.487 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:159: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-12T19:11:38.595 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (a,b,c,d) after stopping [e] 2026-02-12T19:11:38.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:160: TEST_5_mons_checks: ceph mon ok-to-stop a b 2026-02-12T19:11:38.724 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (c,d,e) after stopping [a,b] 2026-02-12T19:11:38.732 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:161: TEST_5_mons_checks: ceph mon ok-to-stop c d 2026-02-12T19:11:38.850 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (a,b,e) after stopping [c,d] 2026-02-12T19:11:38.861 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:162: TEST_5_mons_checks: ceph mon ok-to-stop a b c 2026-02-12T19:11:38.948 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (d,e) after stopping mons [a,b,c] 2026-02-12T19:11:38.950 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:164: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-12T19:11:42.057 INFO:tasks.workunit.client.0.trial010.stderr:adding another mon that is not yet online will not break quorum 2026-02-12T19:11:42.066 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:166: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-12T19:11:42.182 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.a 2026-02-12T19:11:42.190 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:167: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-12T19:11:42.312 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.b 2026-02-12T19:11:42.320 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:168: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-12T19:11:42.428 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.c 2026-02-12T19:11:42.436 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:169: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-12T19:11:42.554 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.d 2026-02-12T19:11:42.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:170: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-12T19:11:42.680 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.e 2026-02-12T19:11:42.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:172: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.a 2026-02-12T19:11:42.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:11:42.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:11:42.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:11:42.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:11:42.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:11:42.797 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:11:42.797 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:173: TEST_5_mons_checks: wait_for_quorum 60 4 2026-02-12T19:11:42.797 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-12T19:11:42.797 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=4 2026-02-12T19:11:42.798 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-12T19:11:42.798 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 4 ]] 2026-02-12T19:11:42.798 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-12T19:11:42.798 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-12T19:11:42.799 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770923562 2026-02-12T19:11:42.799 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:42.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923502 -lt 1770923562 ]] 2026-02-12T19:11:42.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-12T19:11:42.800 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:45.913 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:45.914 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:45.915 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-12T19:11:45.915 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:45.923 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:45.923 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:45.924 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:45.925 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923505 -lt 1770923562 ]] 2026-02-12T19:11:45.925 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-12T19:11:45.925 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:46.041 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:46.041 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.042 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.042 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-12T19:11:46.051 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:46.051 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:46.052 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:46.052 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923506 -lt 1770923562 ]] 2026-02-12T19:11:46.053 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-12T19:11:46.053 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:46.170 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:46.170 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.171 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-12T19:11:46.171 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.180 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:46.180 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:46.180 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:46.181 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923506 -lt 1770923562 ]] 2026-02-12T19:11:46.182 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-12T19:11:46.182 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:46.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:46.292 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.292 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-12T19:11:46.293 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:46.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:46.303 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:46.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923506 -lt 1770923562 ]] 2026-02-12T19:11:46.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-12T19:11:46.304 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:46.418 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:46.418 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.419 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-12T19:11:46.419 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.428 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:46.428 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:46.428 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:46.429 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923506 -lt 1770923562 ]] 2026-02-12T19:11:46.429 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-12T19:11:46.430 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:46.545 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:46.545 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.546 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.546 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-12T19:11:46.555 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:46.555 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:46.556 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:46.557 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923506 -lt 1770923562 ]] 2026-02-12T19:11:46.557 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-12T19:11:46.557 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:46.668 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:46.668 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.669 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-12T19:11:46.669 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.678 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:46.678 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:46.679 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:46.680 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923506 -lt 1770923562 ]] 2026-02-12T19:11:46.680 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-12T19:11:46.680 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:46.793 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:46.793 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.794 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-12T19:11:46.794 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:46.803 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:46.803 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:46.803 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:46.804 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923506 -lt 1770923562 ]] 2026-02-12T19:11:46.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-12T19:11:46.805 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:11:49.918 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:11:49.919 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:49.919 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":8,"quorum":[0,1,2,3,4],"quorum_names":["a","b","c","d","e"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:11:49.920 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-12T19:11:49.930 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:11:49.930 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:11:49.930 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:11:49.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923509 -lt 1770923562 ]] 2026-02-12T19:11:49.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 4' 2026-02-12T19:11:49.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:12:02.052 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:12:02.052 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:02.053 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 4' 2026-02-12T19:12:02.053 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":4,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:02.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-12T19:12:02.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-12T19:12:02.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-12T19:12:02.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-12T19:12:02.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-12T19:12:02.063 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:175: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-12T19:12:02.171 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (b,c,d,e) after stopping [a] 2026-02-12T19:12:02.179 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:176: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-12T19:12:05.318 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (c,d,e) after stopping [b] 2026-02-12T19:12:05.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:177: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-12T19:12:05.429 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (b,d,e) after stopping [c] 2026-02-12T19:12:05.440 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:178: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-12T19:12:05.546 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (b,c,e) after stopping [d] 2026-02-12T19:12:05.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:179: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-12T19:12:08.660 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-12T19:12:08.670 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:181: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-12T19:12:08.775 INFO:tasks.workunit.client.0.trial010.stderr:adding another mon that is not yet online will not break quorum 2026-02-12T19:12:08.783 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:183: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-12T19:12:08.898 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.a 2026-02-12T19:12:08.908 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:184: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-12T19:12:09.015 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.b 2026-02-12T19:12:09.023 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:185: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-12T19:12:09.143 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.c 2026-02-12T19:12:09.151 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:186: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-12T19:12:09.267 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.d 2026-02-12T19:12:09.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:187: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-12T19:12:12.380 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.e 2026-02-12T19:12:12.390 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:189: TEST_5_mons_checks: kill_daemons td/ok-to-stop KILL mon.e 2026-02-12T19:12:12.391 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:12.391 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:12.391 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:12.391 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:12.391 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:12.499 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:12.499 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:190: TEST_5_mons_checks: wait_for_quorum 60 3 2026-02-12T19:12:12.499 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1046: wait_for_quorum: local timeout=60 2026-02-12T19:12:12.499 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1047: wait_for_quorum: local quorumsize=3 2026-02-12T19:12:12.499 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1049: wait_for_quorum: [[ -z 60 ]] 2026-02-12T19:12:12.499 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1053: wait_for_quorum: [[ -z 3 ]] 2026-02-12T19:12:12.499 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1058: wait_for_quorum: no_quorum=1 2026-02-12T19:12:12.499 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: date +%s 2026-02-12T19:12:12.500 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1059: wait_for_quorum: wait_until=1770923592 2026-02-12T19:12:12.501 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:12:12.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923532 -lt 1770923592 ]] 2026-02-12T19:12:12.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:12:12.502 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:12:12.617 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:12:12.617 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:12.617 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":14,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:12.618 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:12:12.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:12:12.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:12:12.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:12:12.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923532 -lt 1770923592 ]] 2026-02-12T19:12:12.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:12:12.630 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:12:15.747 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:12:15.747 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:15.748 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":18,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:15.748 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:12:15.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:12:15.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:12:15.759 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:12:15.760 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923535 -lt 1770923592 ]] 2026-02-12T19:12:15.760 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:12:15.760 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:12:18.879 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:12:18.879 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:18.880 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:18.880 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:12:18.889 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:12:18.889 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:12:18.889 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:12:18.890 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923538 -lt 1770923592 ]] 2026-02-12T19:12:18.891 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:12:18.891 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:12:22.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:12:22.009 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:22.009 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:12:22.009 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:22.019 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:12:22.019 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:12:22.019 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:12:22.020 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923542 -lt 1770923592 ]] 2026-02-12T19:12:22.020 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:12:22.021 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:12:22.132 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:12:22.132 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:22.133 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:12:22.133 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":10,"quorum":[1,2,3,4],"quorum_names":["b","c","d","e"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:22.143 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=false 2026-02-12T19:12:22.143 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ false == \t\r\u\e ]] 2026-02-12T19:12:22.143 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: date +%s 2026-02-12T19:12:22.144 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1060: wait_for_quorum: [[ 1770923542 -lt 1770923592 ]] 2026-02-12T19:12:22.144 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1061: wait_for_quorum: jqfilter='.quorum | length == 3' 2026-02-12T19:12:22.145 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: timeout 60 ceph quorum_status --format=json 2026-02-12T19:12:28.263 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1062: wait_for_quorum: jqinput=' 2026-02-12T19:12:28.263 INFO:tasks.workunit.client.0.trial010.stderr:{"election_epoch":14,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:28.263 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: echo '{"election_epoch":14,"quorum":[1,2,3],"quorum_names":["b","c","d"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4544132024016699391","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"]},"monmap":{"epoch":1,"fsid":"9dfe8ccd-165d-45ee-a6bb-cf436306c0de","modified":"2026-02-12T19:11:30.968596Z","created":"2026-02-12T19:11:30.968596Z","min_mon_release":20,"min_mon_release_name":"tentacle","election_strategy":1,"disallowed_leaders":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid","tentacle","nvmeof_beacon_diff"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7150","nonce":0}]},"addr":"127.0.0.1:7150/0","public_addr":"127.0.0.1:7150/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968591+0000","crush_location":[]},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7151","nonce":0}]},"addr":"127.0.0.1:7151/0","public_addr":"127.0.0.1:7151/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7152","nonce":0}]},"addr":"127.0.0.1:7152/0","public_addr":"127.0.0.1:7152/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968593+0000","crush_location":[]},{"rank":3,"name":"d","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7153","nonce":0}]},"addr":"127.0.0.1:7153/0","public_addr":"127.0.0.1:7153/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968594+0000","crush_location":[]},{"rank":4,"name":"e","public_addrs":{"addrvec":[{"type":"v2","addr":"127.0.0.1:7154","nonce":0}]},"addr":"127.0.0.1:7154/0","public_addr":"127.0.0.1:7154/0","priority":0,"weight":0,"time_added":"2026-02-12T19:11:30:968595+0000","crush_location":[]}]}}' 2026-02-12T19:12:28.263 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: jq '.quorum | length == 3' 2026-02-12T19:12:28.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1063: wait_for_quorum: res=true 2026-02-12T19:12:28.273 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1064: wait_for_quorum: [[ true == \t\r\u\e ]] 2026-02-12T19:12:28.273 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1065: wait_for_quorum: no_quorum=0 2026-02-12T19:12:28.273 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1066: wait_for_quorum: break 2026-02-12T19:12:28.273 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1069: wait_for_quorum: return 0 2026-02-12T19:12:28.273 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:192: TEST_5_mons_checks: ceph mon ok-to-stop a 2026-02-12T19:12:28.374 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (b,c,d) after stopping [a] 2026-02-12T19:12:28.381 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:193: TEST_5_mons_checks: ceph mon ok-to-stop b 2026-02-12T19:12:28.475 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (c,d) after stopping mons [b] 2026-02-12T19:12:28.477 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:194: TEST_5_mons_checks: ceph mon ok-to-stop c 2026-02-12T19:12:28.561 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (b,d) after stopping mons [c] 2026-02-12T19:12:28.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:195: TEST_5_mons_checks: ceph mon ok-to-stop d 2026-02-12T19:12:28.649 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: not enough monitors would be available (b,c) after stopping mons [d] 2026-02-12T19:12:28.652 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:196: TEST_5_mons_checks: ceph mon ok-to-stop e 2026-02-12T19:12:28.754 INFO:tasks.workunit.client.0.trial010.stderr:quorum should be preserved (b,c,d) after stopping [e] 2026-02-12T19:12:28.762 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:198: TEST_5_mons_checks: ceph mon ok-to-add-offline 2026-02-12T19:12:28.860 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: adding a monitor may break quorum (until that monitor starts) 2026-02-12T19:12:28.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:200: TEST_5_mons_checks: ceph mon ok-to-rm a 2026-02-12T19:12:28.967 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.a 2026-02-12T19:12:28.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:201: TEST_5_mons_checks: ceph mon ok-to-rm b 2026-02-12T19:12:29.066 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: removing mon.b would break quorum 2026-02-12T19:12:29.070 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:202: TEST_5_mons_checks: ceph mon ok-to-rm c 2026-02-12T19:12:29.156 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: removing mon.c would break quorum 2026-02-12T19:12:29.159 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:203: TEST_5_mons_checks: ceph mon ok-to-rm d 2026-02-12T19:12:29.244 INFO:tasks.workunit.client.0.trial010.stderr:Error EBUSY: removing mon.d would break quorum 2026-02-12T19:12:29.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:204: TEST_5_mons_checks: ceph mon ok-to-rm e 2026-02-12T19:12:29.351 INFO:tasks.workunit.client.0.trial010.stderr:safe to remove mon.e 2026-02-12T19:12:29.358 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:22: run: kill_daemons td/ok-to-stop KILL 2026-02-12T19:12:29.359 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:29.359 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:29.359 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:29.360 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:29.360 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:29.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:29.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/ok-to-stop.sh:23: run: teardown td/ok-to-stop 2026-02-12T19:12:29.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:12:29.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:12:29.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:12:29.469 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:29.469 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:29.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:29.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:29.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:29.476 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:29.476 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:12:29.477 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:12:29.477 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:12:29.479 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:12:29.479 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:12:29.479 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:12:29.480 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:29.480 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:12:29.481 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:12:29.481 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:29.482 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:12:29.483 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:12:29.483 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:12:29.498 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:12:29.499 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:29.499 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:12:29.499 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:12:29.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:12:29.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:12:29.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-12T19:12:29.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/ok-to-stop 0 2026-02-12T19:12:29.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ok-to-stop 2026-02-12T19:12:29.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-12T19:12:29.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ok-to-stop KILL 2026-02-12T19:12:29.501 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:29.502 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:29.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:29.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:29.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:29.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:29.504 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:12:29.505 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:12:29.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:12:29.506 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:12:29.506 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:12:29.506 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:12:29.507 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:29.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:12:29.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:12:29.508 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:29.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:12:29.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-12T19:12:29.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ok-to-stop 2026-02-12T19:12:29.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:12:29.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:29.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.31636 2026-02-12T19:12:29.512 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.31636 2026-02-12T19:12:29.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:12:29.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:12:29.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-12T19:12:29.515 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-12T19:12:29.515 DEBUG:teuthology.orchestra.run.trial010:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-12T19:12:29.564 INFO:tasks.workunit:Running workunit misc/rados-striper.sh... 2026-02-12T19:12:29.564 DEBUG:teuthology.orchestra.run.trial010:workunit test misc/rados-striper.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh 2026-02-12T19:12:29.617 INFO:tasks.workunit.client.0.trial010.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2393: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PYTHONWARNINGS=ignore 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PYTHONWARNINGS=ignore 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export CEPH_CONF=/dev/null 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: CEPH_CONF=/dev/null 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: unset CEPH_ARGS 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2398: main: local code 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: run td/rados-striper 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:20: run: local dir=td/rados-striper 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:21: run: shift 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: export CEPH_MON=127.0.0.1:7116 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:23: run: CEPH_MON=127.0.0.1:7116 2026-02-12T19:12:29.620 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:24: run: export CEPH_ARGS 2026-02-12T19:12:29.621 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: uuidgen 2026-02-12T19:12:29.621 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:25: run: CEPH_ARGS+='--fsid=8558dfb1-5113-4dff-9430-abed32fcb133 --auth-supported=none ' 2026-02-12T19:12:29.621 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:26: run: CEPH_ARGS+='--mon-host=127.0.0.1:7116 ' 2026-02-12T19:12:29.621 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:29: run: setup td/rados-striper 2026-02-12T19:12:29.621 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/rados-striper 2026-02-12T19:12:29.621 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/rados-striper 2026-02-12T19:12:29.621 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-12T19:12:29.621 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:12:29.621 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-12T19:12:29.622 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:29.622 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:29.622 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:29.622 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:29.622 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:29.624 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:29.624 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:12:29.624 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:12:29.625 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:12:29.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:12:29.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:12:29.626 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:12:29.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:29.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:12:29.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:12:29.627 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:29.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:12:29.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:12:29.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-12T19:12:29.630 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:12:29.630 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:29.631 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.47027 2026-02-12T19:12:29.631 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.47027 2026-02-12T19:12:29.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:12:29.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:12:29.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/rados-striper 2026-02-12T19:12:29.633 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:12:29.633 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:29.633 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.47027 2026-02-12T19:12:29.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.47027 2026-02-12T19:12:29.635 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:12:29.635 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-12T19:12:29.635 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-12T19:12:29.635 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:12:29.635 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/rados-striper 1' TERM HUP INT 2026-02-12T19:12:29.635 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:32: run: run_mon td/rados-striper a 2026-02-12T19:12:29.635 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/rados-striper 2026-02-12T19:12:29.636 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:12:29.636 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:12:29.636 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:12:29.636 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/rados-striper/a 2026-02-12T19:12:29.636 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/rados-striper/a --run-dir=td/rados-striper 2026-02-12T19:12:29.657 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:12:29.657 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:29.657 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:29.657 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:29.657 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:29.657 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.47027 2026-02-12T19:12:29.658 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.47027/$cluster-$name.asok' 2026-02-12T19:12:29.658 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/rados-striper/a '--log-file=td/rados-striper/$name.log' '--admin-socket=/tmp/ceph-asok.47027/$cluster-$name.asok' --mon-cluster-log-file=td/rados-striper/log --run-dir=td/rados-striper '--pid-file=td/rados-striper/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:12:29.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:12:29.687 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:12:29.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:12:29.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:12:29.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:12:29.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:12:29.688 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:12:29.688 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:12:29.688 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:12:29.689 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:29.689 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:29.689 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.47027 2026-02-12T19:12:29.689 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.47027/ceph-mon.a.asok 2026-02-12T19:12:29.690 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:12:29.690 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.47027/ceph-mon.a.asok config get fsid 2026-02-12T19:12:29.745 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:12:29.745 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:12:29.745 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:12:29.745 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:12:29.745 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:12:29.747 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:12:29.747 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:12:29.747 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:12:29.747 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:29.747 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:29.747 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.47027 2026-02-12T19:12:29.747 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.47027/ceph-mon.a.asok 2026-02-12T19:12:29.747 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:12:29.747 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.47027/ceph-mon.a.asok config get mon_host 2026-02-12T19:12:29.804 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:33: run: run_osd td/rados-striper 0 2026-02-12T19:12:29.804 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-12T19:12:29.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:12:29.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:12:29.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:12:29.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/0 2026-02-12T19:12:29.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8558dfb1-5113-4dff-9430-abed32fcb133 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-12T19:12:29.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:12:29.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:12:29.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:12:29.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/0' 2026-02-12T19:12:29.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/0/journal' 2026-02-12T19:12:29.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.47027 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.47027/$cluster-$name.asok' 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47027/$cluster-$name.asok' 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:12:29.806 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:12:29.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-12T19:12:29.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-12T19:12:29.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:12:29.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:12:29.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:12:29.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:12:29.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:12:29.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:12:29.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/0 2026-02-12T19:12:29.807 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:12:29.808 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f1442c00-e158-4421-8261-52205bbc2700 2026-02-12T19:12:29.808 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f1442c00-e158-4421-8261-52205bbc2700' 2026-02-12T19:12:29.808 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 f1442c00-e158-4421-8261-52205bbc2700 2026-02-12T19:12:29.809 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:12:29.817 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAdJo5pUe57MBAAFrfKMiSUOXp+xghjf/giMw== 2026-02-12T19:12:29.817 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAdJo5pUe57MBAAFrfKMiSUOXp+xghjf/giMw=="}' 2026-02-12T19:12:29.817 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f1442c00-e158-4421-8261-52205bbc2700 -i td/rados-striper/0/new.json 2026-02-12T19:12:29.937 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:29.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/0/new.json 2026-02-12T19:12:29.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=8558dfb1-5113-4dff-9430-abed32fcb133 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.47027/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAdJo5pUe57MBAAFrfKMiSUOXp+xghjf/giMw== --osd-uuid f1442c00-e158-4421-8261-52205bbc2700 2026-02-12T19:12:29.962 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:29.955+0000 7f39074a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:29.963 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:29.955+0000 7f39074a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:29.964 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:29.959+0000 7f39074a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:29.965 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:29.959+0000 7f39074a4a40 -1 bdev(0x5635eb4a2400 td/rados-striper/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:12:29.965 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:29.959+0000 7f39074a4a40 -1 bluestore(td/rados-striper/0) _read_fsid unparsable uuid 2026-02-12T19:12:30.650 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/0/keyring 2026-02-12T19:12:30.650 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:12:30.651 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:12:30.651 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:12:30.652 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:12:30.774 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:12:30.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:12:30.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=8558dfb1-5113-4dff-9430-abed32fcb133 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/0 --osd-journal=td/rados-striper/0/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.47027/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:12:30.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:12:30.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:12:30.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:12:30.790 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:30.783+0000 7f59e7061a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:30.791 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:30.787+0000 7f59e7061a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:30.792 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:30.787+0000 7f59e7061a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:30.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:12:30.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:12:30.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:12:30.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:12:30.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:12:30.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:30.882 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:30.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:12:30.883 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:30.883 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:30.992 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:31.221 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:31.215+0000 7f59e7061a40 -1 Falling back to public interface 2026-02-12T19:12:31.869 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:31.863+0000 7f59e7061a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:12:31.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:31.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:31.993 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:12:31.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:12:31.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:31.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:32.103 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:33.105 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:33.105 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:33.105 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:12:33.105 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:12:33.106 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:33.106 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:33.215 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3448972137,v1:127.0.0.1:6801/3448972137] [v2:127.0.0.1:6802/3448972137,v1:127.0.0.1:6803/3448972137] exists,up f1442c00-e158-4421-8261-52205bbc2700 2026-02-12T19:12:33.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:34: run: run_osd td/rados-striper 1 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/1 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8558dfb1-5113-4dff-9430-abed32fcb133 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:12:33.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/1' 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/1/journal' 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.47027 2026-02-12T19:12:33.220 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.47027/$cluster-$name.asok' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47027/$cluster-$name.asok' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:12:33.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/1 2026-02-12T19:12:33.222 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:12:33.222 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ca1c25b1-05e3-4740-a78b-8e3c56491b62 2026-02-12T19:12:33.223 INFO:tasks.workunit.client.0.trial010.stdout:add osd1 ca1c25b1-05e3-4740-a78b-8e3c56491b62 2026-02-12T19:12:33.223 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 ca1c25b1-05e3-4740-a78b-8e3c56491b62' 2026-02-12T19:12:33.223 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:12:33.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAhJo5pCw+nDRAAjsc9dbABunRTNwg1ejgImg== 2026-02-12T19:12:33.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAhJo5pCw+nDRAAjsc9dbABunRTNwg1ejgImg=="}' 2026-02-12T19:12:33.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ca1c25b1-05e3-4740-a78b-8e3c56491b62 -i td/rados-striper/1/new.json 2026-02-12T19:12:33.343 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:12:33.351 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/1/new.json 2026-02-12T19:12:33.351 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=8558dfb1-5113-4dff-9430-abed32fcb133 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.47027/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAhJo5pCw+nDRAAjsc9dbABunRTNwg1ejgImg== --osd-uuid ca1c25b1-05e3-4740-a78b-8e3c56491b62 2026-02-12T19:12:33.368 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:33.362+0000 7f0acc585a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:33.369 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:33.362+0000 7f0acc585a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:33.370 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:33.362+0000 7f0acc585a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:33.371 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:33.362+0000 7f0acc585a40 -1 bdev(0x558c47724400 td/rados-striper/1/block) open stat got: (1) Operation not permitted 2026-02-12T19:12:33.371 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:33.362+0000 7f0acc585a40 -1 bluestore(td/rados-striper/1) _read_fsid unparsable uuid 2026-02-12T19:12:34.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/1/keyring 2026-02-12T19:12:34.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:12:34.016 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T19:12:34.016 INFO:tasks.workunit.client.0.trial010.stdout:adding osd1 key to auth repository 2026-02-12T19:12:34.016 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:12:34.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T19:12:34.138 INFO:tasks.workunit.client.0.trial010.stdout:start osd.1 2026-02-12T19:12:34.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=8558dfb1-5113-4dff-9430-abed32fcb133 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/1 --osd-journal=td/rados-striper/1/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.47027/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:12:34.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:12:34.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:12:34.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:12:34.153 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:34.146+0000 7f2b6c70ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:34.155 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:34.146+0000 7f2b6c70ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:34.156 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:34.150+0000 7f2b6c70ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:34.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T19:12:34.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:12:34.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T19:12:34.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:12:34.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:12:34.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:34.251 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:34.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:12:34.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:34.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:12:34.358 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:34.513 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:34.506+0000 7f2b6c70ca40 -1 Falling back to public interface 2026-02-12T19:12:35.073 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:35.066+0000 7f2b6c70ca40 -1 osd.1 0 log_to_monitors true 2026-02-12T19:12:35.359 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:35.360 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:35.360 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:12:35.360 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:12:35.360 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:35.361 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:12:35.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:36.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:36.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:36.472 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:12:36.473 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:12:36.473 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:36.473 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:12:36.582 INFO:tasks.workunit.client.0.trial010.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6808/2117267293,v1:127.0.0.1:6809/2117267293] [v2:127.0.0.1:6810/2117267293,v1:127.0.0.1:6811/2117267293] exists,up ca1c25b1-05e3-4740-a78b-8e3c56491b62 2026-02-12T19:12:36.585 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:12:36.585 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:12:36.585 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:12:36.585 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:35: run: run_osd td/rados-striper 2 2026-02-12T19:12:36.585 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/rados-striper 2026-02-12T19:12:36.585 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:12:36.585 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-12T19:12:36.585 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/rados-striper/2 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=8558dfb1-5113-4dff-9430-abed32fcb133 --auth-supported=none --mon-host=127.0.0.1:7116 ' 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/rados-striper/2' 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/rados-striper/2/journal' 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/rados-striper' 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:36.586 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.47027 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.47027/$cluster-$name.asok' 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.47027/$cluster-$name.asok' 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/rados-striper/$name.log' 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/rados-striper/$name.pid' 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:12:36.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:12:36.588 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:12:36.588 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:12:36.588 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:12:36.588 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:12:36.588 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/rados-striper/2 2026-02-12T19:12:36.588 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:12:36.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=956dd146-8aee-48e5-963f-9cd60b9a5028 2026-02-12T19:12:36.589 INFO:tasks.workunit.client.0.trial010.stdout:add osd2 956dd146-8aee-48e5-963f-9cd60b9a5028 2026-02-12T19:12:36.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 956dd146-8aee-48e5-963f-9cd60b9a5028' 2026-02-12T19:12:36.589 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:12:36.597 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAkJo5p8NJpIxAA3DQbWIQSU5IjhuWhalDenQ== 2026-02-12T19:12:36.598 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAkJo5p8NJpIxAA3DQbWIQSU5IjhuWhalDenQ=="}' 2026-02-12T19:12:36.598 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 956dd146-8aee-48e5-963f-9cd60b9a5028 -i td/rados-striper/2/new.json 2026-02-12T19:12:36.717 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:12:36.725 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/rados-striper/2/new.json 2026-02-12T19:12:36.726 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=8558dfb1-5113-4dff-9430-abed32fcb133 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.47027/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAkJo5p8NJpIxAA3DQbWIQSU5IjhuWhalDenQ== --osd-uuid 956dd146-8aee-48e5-963f-9cd60b9a5028 2026-02-12T19:12:36.741 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:36.734+0000 7f7b6fc60a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:36.743 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:36.734+0000 7f7b6fc60a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:36.744 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:36.738+0000 7f7b6fc60a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:36.745 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:36.738+0000 7f7b6fc60a40 -1 bdev(0x558cce246400 td/rados-striper/2/block) open stat got: (1) Operation not permitted 2026-02-12T19:12:36.745 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:36.738+0000 7f7b6fc60a40 -1 bluestore(td/rados-striper/2) _read_fsid unparsable uuid 2026-02-12T19:12:37.430 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/rados-striper/2/keyring 2026-02-12T19:12:37.431 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:12:37.431 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-12T19:12:37.432 INFO:tasks.workunit.client.0.trial010.stdout:adding osd2 key to auth repository 2026-02-12T19:12:37.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/rados-striper/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:12:37.555 INFO:tasks.workunit.client.0.trial010.stdout:start osd.2 2026-02-12T19:12:37.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-12T19:12:37.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=8558dfb1-5113-4dff-9430-abed32fcb133 --auth-supported=none --mon-host=127.0.0.1:7116 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/rados-striper/2 --osd-journal=td/rados-striper/2/journal --chdir= --run-dir=td/rados-striper '--admin-socket=/tmp/ceph-asok.47027/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/rados-striper/$name.log' '--pid-file=td/rados-striper/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:12:37.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:12:37.557 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:12:37.557 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:12:37.570 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:37.562+0000 7ff2c94eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:37.572 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:37.566+0000 7ff2c94eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:37.573 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:37.566+0000 7ff2c94eea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:37.663 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-12T19:12:37.664 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:12:37.664 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-12T19:12:37.664 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:12:37.664 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:12:37.664 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:37.664 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:37.665 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:12:37.665 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:37.665 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T19:12:37.772 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:37.941 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:37.934+0000 7ff2c94eea40 -1 Falling back to public interface 2026-02-12T19:12:38.517 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:38.510+0000 7ff2c94eea40 -1 osd.2 0 log_to_monitors true 2026-02-12T19:12:38.773 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:38.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:38.774 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:12:38.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:12:38.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:38.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T19:12:38.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:39.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:39.889 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:39.889 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:12:39.889 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:12:39.889 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:39.890 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-12T19:12:40.000 INFO:tasks.workunit.client.0.trial010.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6816/1684579496,v1:127.0.0.1:6817/1684579496] [v2:127.0.0.1:6818/1684579496,v1:127.0.0.1:6819/1684579496] exists,up 956dd146-8aee-48e5-963f-9cd60b9a5028 2026-02-12T19:12:40.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:12:40.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:12:40.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:12:40.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:36: run: create_rbd_pool 2026-02-12T19:12:40.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:12:40.102 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:12:40.112 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:12:40.112 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:12:40.243 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:12:40.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:12:41.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:12:41.532 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:39: run: dd if=/dev/urandom of=td/rados-striper/toyfile bs=1234 count=1 2026-02-12T19:12:41.533 INFO:tasks.workunit.client.0.trial010.stderr:1+0 records in 2026-02-12T19:12:41.533 INFO:tasks.workunit.client.0.trial010.stderr:1+0 records out 2026-02-12T19:12:41.533 INFO:tasks.workunit.client.0.trial010.stderr:1234 bytes (1.2 kB, 1.2 KiB) copied, 5.934e-05 s, 20.8 MB/s 2026-02-12T19:12:41.533 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:42: run: rados --pool rbd --striper put toyfile td/rados-striper/toyfile 2026-02-12T19:12:41.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: rados --pool rbd --striper stat toyfile 2026-02-12T19:12:41.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:45: run: cut -d , -f 2 2026-02-12T19:12:41.584 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-12T19:12:41.585 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:46: run: cut -d , -f 2 2026-02-12T19:12:41.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:47: run: echo ' size 1234' 2026-02-12T19:12:41.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:48: run: diff -w td/rados-striper/stripedStat td/rados-striper/refstat 2026-02-12T19:12:41.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:49: run: diff -w td/rados-striper/stat td/rados-striper/refstat 2026-02-12T19:12:41.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:50: run: rados --pool rbd stat toyfile 2026-02-12T19:12:41.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:51: run: grep -q 'No such file or directory' td/rados-striper/staterror 2026-02-12T19:12:41.634 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:54: run: rados --pool rbd --striper get toyfile td/rados-striper/stripedGroup 2026-02-12T19:12:41.658 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:55: run: diff -w td/rados-striper/toyfile td/rados-striper/stripedGroup 2026-02-12T19:12:41.659 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:56: run: rados --pool rbd get toyfile.0000000000000000 td/rados-striper/nonSTripedGroup 2026-02-12T19:12:41.681 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:57: run: diff -w td/rados-striper/toyfile td/rados-striper/nonSTripedGroup 2026-02-12T19:12:41.682 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:60: run: rados --pool rbd --striper truncate toyfile 12 2026-02-12T19:12:41.704 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: rados --pool rbd --striper stat toyfile 2026-02-12T19:12:41.704 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:61: run: cut -d , -f 2 2026-02-12T19:12:41.721 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: rados --pool rbd stat toyfile.0000000000000000 2026-02-12T19:12:41.721 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:62: run: cut -d , -f 2 2026-02-12T19:12:41.743 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:63: run: echo ' size 12' 2026-02-12T19:12:41.743 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:64: run: diff -w td/rados-striper/stripedStat td/rados-striper/reftrunc 2026-02-12T19:12:41.744 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:65: run: diff -w td/rados-striper/stat td/rados-striper/reftrunc 2026-02-12T19:12:41.745 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:69: run: rados --pool rbd --striper setxattr toyfile somexattr somevalue 2026-02-12T19:12:41.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:70: run: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-12T19:12:41.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:71: run: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-12T19:12:41.814 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:72: run: echo somevalue 2026-02-12T19:12:41.814 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:73: run: diff -w td/rados-striper/xattrvalue td/rados-striper/refvalue 2026-02-12T19:12:41.815 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:74: run: diff -w td/rados-striper/xattrvalue2 td/rados-striper/refvalue 2026-02-12T19:12:41.816 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:75: run: rados --pool rbd --striper listxattr toyfile 2026-02-12T19:12:41.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:76: run: echo somexattr 2026-02-12T19:12:41.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:77: run: diff -w td/rados-striper/xattrlist td/rados-striper/reflist 2026-02-12T19:12:41.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: rados --pool rbd listxattr toyfile.0000000000000000 2026-02-12T19:12:41.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:78: run: grep -v striper 2026-02-12T19:12:41.861 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:79: run: diff -w td/rados-striper/xattrlist2 td/rados-striper/reflist 2026-02-12T19:12:41.862 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:80: run: rados --pool rbd --striper rmxattr toyfile somexattr 2026-02-12T19:12:41.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:82: run: local 'attr_not_found_str=No data available' 2026-02-12T19:12:41.886 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: uname 2026-02-12T19:12:41.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:83: run: '[' Linux = FreeBSD ']' 2026-02-12T19:12:41.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:85: run: expect_failure td/rados-striper 'No data available' rados --pool rbd --striper getxattr toyfile somexattr 2026-02-12T19:12:41.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-12T19:12:41.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-12T19:12:41.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-12T19:12:41.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-12T19:12:41.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-12T19:12:41.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper getxattr toyfile somexattr 2026-02-12T19:12:41.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-12T19:12:41.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-12T19:12:41.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-12T19:12:41.907 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-12T19:12:41.907 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:87: run: expect_failure td/rados-striper 'No data available' rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-12T19:12:41.907 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-12T19:12:41.907 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-12T19:12:41.907 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No data available' 2026-02-12T19:12:41.907 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-12T19:12:41.907 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-12T19:12:41.907 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd getxattr toyfile.0000000000000000 somexattr 2026-02-12T19:12:41.929 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-12T19:12:41.929 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-12T19:12:41.929 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No data available' td/rados-striper/out 2026-02-12T19:12:41.930 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-12T19:12:41.930 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:91: run: rados --pool rbd --striper rm toyfile 2026-02-12T19:12:41.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:92: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd --striper stat toyfile 2026-02-12T19:12:41.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-12T19:12:41.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-12T19:12:41.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-12T19:12:41.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-12T19:12:41.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-12T19:12:41.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd --striper stat toyfile 2026-02-12T19:12:41.978 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-12T19:12:41.978 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-12T19:12:41.978 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-12T19:12:41.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-12T19:12:41.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:94: run: expect_failure td/rados-striper 'No such file or directory' rados --pool rbd stat toyfile.0000000000000000 2026-02-12T19:12:41.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/rados-striper 2026-02-12T19:12:41.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-12T19:12:41.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local 'expected=No such file or directory' 2026-02-12T19:12:41.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-12T19:12:41.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-12T19:12:41.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: rados --pool rbd stat toyfile.0000000000000000 2026-02-12T19:12:42.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-12T19:12:42.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-12T19:12:42.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet 'No such file or directory' td/rados-striper/out 2026-02-12T19:12:42.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-12T19:12:42.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/rados-striper.sh:98: run: teardown td/rados-striper 2026-02-12T19:12:42.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-12T19:12:42.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:12:42.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-12T19:12:42.001 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:42.001 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:42.002 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:42.002 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:42.002 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:42.110 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:42.110 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:12:42.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:12:42.111 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:12:42.112 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:12:42.112 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:12:42.113 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:12:42.114 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:42.114 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:12:42.114 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:12:42.114 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:42.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:12:42.117 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:12:42.117 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-12T19:12:42.124 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:12:42.124 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:42.124 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.47027 2026-02-12T19:12:42.125 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.47027 2026-02-12T19:12:42.126 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:12:42.126 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:12:42.126 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: code=0 2026-02-12T19:12:42.126 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2404: main: teardown td/rados-striper 0 2026-02-12T19:12:42.126 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/rados-striper 2026-02-12T19:12:42.126 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-12T19:12:42.126 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/rados-striper KILL 2026-02-12T19:12:42.126 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:42.126 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:42.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:42.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:42.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:42.129 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:42.129 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:12:42.130 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:12:42.131 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:12:42.132 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:12:42.132 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:12:42.132 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:12:42.133 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:42.133 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:12:42.134 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:12:42.134 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:42.135 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:12:42.136 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-12T19:12:42.136 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/rados-striper 2026-02-12T19:12:42.137 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:12:42.137 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:42.137 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.47027 2026-02-12T19:12:42.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.47027 2026-02-12T19:12:42.139 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:12:42.139 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:12:42.139 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: return 0 2026-02-12T19:12:42.140 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-12T19:12:42.140 DEBUG:teuthology.orchestra.run.trial010:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-12T19:12:42.197 INFO:tasks.workunit:Running workunit misc/test-ceph-helpers.sh... 2026-02-12T19:12:42.197 DEBUG:teuthology.orchestra.run.trial010:workunit test misc/test-ceph-helpers.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh 2026-02-12T19:12:42.261 INFO:tasks.workunit.client.0.trial010.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-12T19:12:42.264 INFO:tasks.workunit.client.0.trial010.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-12T19:12:42.264 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2414: run_tests: export PATH=./bin:.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-12T19:12:42.264 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2414: run_tests: PATH=./bin:.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-12T19:12:42.264 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2416: run_tests: export CEPH_MON=127.0.0.1:7109 2026-02-12T19:12:42.264 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2416: run_tests: CEPH_MON=127.0.0.1:7109 2026-02-12T19:12:42.264 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2417: run_tests: export CEPH_ARGS 2026-02-12T19:12:42.264 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2418: run_tests: uuidgen 2026-02-12T19:12:42.265 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2418: run_tests: CEPH_ARGS+=' --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none ' 2026-02-12T19:12:42.266 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2419: run_tests: CEPH_ARGS+='--mon-host=127.0.0.1:7109 ' 2026-02-12T19:12:42.266 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2420: run_tests: export CEPH_CONF=/dev/null 2026-02-12T19:12:42.266 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2420: run_tests: CEPH_CONF=/dev/null 2026-02-12T19:12:42.266 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: set 2026-02-12T19:12:42.266 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: sed -n -e 's/^\(test_[0-9a-z_]*\) .*/\1/p' 2026-02-12T19:12:42.268 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2422: run_tests: local 'funcs=test_activate_osd 2026-02-12T19:12:42.268 INFO:tasks.workunit.client.0.trial010.stderr:test_activate_osd_after_mark_down 2026-02-12T19:12:42.268 INFO:tasks.workunit.client.0.trial010.stderr:test_activate_osd_skip_benchmark 2026-02-12T19:12:42.268 INFO:tasks.workunit.client.0.trial010.stderr:test_destroy_osd 2026-02-12T19:12:42.268 INFO:tasks.workunit.client.0.trial010.stderr:test_display_logs 2026-02-12T19:12:42.268 INFO:tasks.workunit.client.0.trial010.stderr:test_erasure_code_plugin_exists 2026-02-12T19:12:42.268 INFO:tasks.workunit.client.0.trial010.stderr:test_expect_failure 2026-02-12T19:12:42.268 INFO:tasks.workunit.client.0.trial010.stderr:test_file_with_random_data 2026-02-12T19:12:42.268 INFO:tasks.workunit.client.0.trial010.stderr:test_flush_pg_stats 2026-02-12T19:12:42.268 INFO:tasks.workunit.client.0.trial010.stderr:test_get_config 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_is_making_recovery_progress 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_last_scrub_stamp 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_not_primary 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_num_active_clean 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_num_active_or_peered 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_num_pgs 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_op_scheduler 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_osd_id_used_by_pgs 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_osds 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_pg 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_primary 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_get_timeout_delays 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_is_clean 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_kill_daemon 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_kill_daemons 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_objectstore_tool 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_pg_schedule_scrub 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_pg_scrub 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_repair 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_run_in_background 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_run_mon 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_run_osd 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_set_config 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_setup 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_teardown 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_wait_background 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_wait_for_clean 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_wait_for_health_ok 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_wait_for_osd 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_wait_for_peered 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_wait_for_scrub 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:test_wait_osd_id_used_by_pgs' 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2423: run_tests: local dir=td/ceph-helpers 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd td/ceph-helpers 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:877: test_activate_osd: local dir=td/ceph-helpers 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: test_activate_osd: setup td/ceph-helpers 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:12:42.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:12:42.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:42.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:42.270 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:42.270 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:42.270 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:42.271 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:42.271 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:12:42.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:12:42.273 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:12:42.274 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:12:42.274 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:12:42.274 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:12:42.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:42.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:12:42.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:12:42.275 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:42.276 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:12:42.278 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:12:42.278 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:12:42.279 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:12:42.279 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:42.279 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:42.279 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:12:42.280 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:12:42.280 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:12:42.280 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:12:42.282 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:12:42.282 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:42.282 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:42.282 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:12:42.283 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:12:42.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-12T19:12:42.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-12T19:12:42.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:12:42.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:12:42.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:881: test_activate_osd: run_mon td/ceph-helpers a 2026-02-12T19:12:42.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:12:42.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:12:42.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:12:42.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:12:42.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:12:42.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-12T19:12:42.307 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:12:42.308 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:42.308 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:42.308 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:42.308 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:42.308 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:42.308 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:42.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:12:42.335 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:12:42.338 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:12:42.339 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:12:42.339 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:12:42.339 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:12:42.339 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:12:42.339 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:12:42.339 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:12:42.339 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:12:42.340 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:42.340 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:42.340 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:42.340 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:12:42.340 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:12:42.340 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:12:42.395 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:12:42.395 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:12:42.396 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:12:42.396 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:12:42.396 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:12:42.396 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:12:42.397 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:12:42.397 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:12:42.397 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:42.398 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:42.398 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:42.398 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:12:42.398 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:12:42.399 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:12:42.446 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: test_activate_osd: run_mgr td/ceph-helpers x 2026-02-12T19:12:42.446 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:12:42.446 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:12:42.447 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:12:42.447 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:12:42.447 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:12:42.447 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:12:42.555 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:12:42.555 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:42.555 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:42.555 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:42.555 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:42.555 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:42.555 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:42.556 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:12:42.556 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:12:42.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:884: test_activate_osd: run_osd td/ceph-helpers 0 2026-02-12T19:12:42.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:12:42.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:12:42.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:12:42.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:12:42.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:12:42.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:12:42.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:42.578 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:12:42.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:12:42.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:12:42.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:12:42.580 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:12:42.580 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=82414629-1177-4cc7-842d-c9e867137252 2026-02-12T19:12:42.581 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 82414629-1177-4cc7-842d-c9e867137252' 2026-02-12T19:12:42.581 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 82414629-1177-4cc7-842d-c9e867137252 2026-02-12T19:12:42.581 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:12:42.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAqJo5py6rpIhAAXxceMijyJCujH1fVlGCI4w== 2026-02-12T19:12:42.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAqJo5py6rpIhAAXxceMijyJCujH1fVlGCI4w=="}' 2026-02-12T19:12:42.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 82414629-1177-4cc7-842d-c9e867137252 -i td/ceph-helpers/0/new.json 2026-02-12T19:12:42.707 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:42.715 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:12:42.716 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAqJo5py6rpIhAAXxceMijyJCujH1fVlGCI4w== --osd-uuid 82414629-1177-4cc7-842d-c9e867137252 2026-02-12T19:12:42.733 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:42.726+0000 7fcc276bca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:42.734 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:42.726+0000 7fcc276bca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:42.735 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:42.726+0000 7fcc276bca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:42.736 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:42.730+0000 7fcc276bca40 -1 bdev(0x55d8c89f6400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:12:42.736 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:42.730+0000 7fcc276bca40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:12:43.475 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:12:43.476 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:12:43.476 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:12:43.477 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:12:43.477 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:12:43.598 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:12:43.599 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:12:43.599 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:12:43.599 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:12:43.599 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:12:43.599 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:12:43.623 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:43.614+0000 7f007296aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:43.626 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:43.618+0000 7f007296aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:43.627 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:43.618+0000 7f007296aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:43.708 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:12:43.708 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:12:43.709 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:12:43.709 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:12:43.709 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:12:43.709 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:43.709 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:43.710 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:12:43.710 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:43.710 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:43.816 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:44.181 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:44.174+0000 7f007296aa40 -1 Falling back to public interface 2026-02-12T19:12:44.818 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:44.818 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:44.819 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:12:44.819 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:12:44.819 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:44.820 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:44.925 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:45.177 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:45.170+0000 7f007296aa40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:12:45.927 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:45.927 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:45.927 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:12:45.927 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:12:45.928 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:45.928 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:46.139 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1164015001,v1:127.0.0.1:6801/1164015001] [v2:127.0.0.1:6802/1164015001,v1:127.0.0.1:6803/1164015001] exists,up 82414629-1177-4cc7-842d-c9e867137252 2026-02-12T19:12:46.140 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:12:46.140 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:12:46.140 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:12:46.141 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: get_asok_path osd.0 2026-02-12T19:12:46.141 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:12:46.141 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:12:46.141 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:46.141 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:46.141 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:46.141 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-osd.0.asok 2026-02-12T19:12:46.142 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: CEPH_ARGS= 2026-02-12T19:12:46.142 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.52248/ceph-osd.0.asok config get osd_max_backfills 2026-02-12T19:12:46.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:886: test_activate_osd: local 'backfills={"osd_max_backfills":"1"}' 2026-02-12T19:12:46.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: echo '{"osd_max_backfills":"1"}' 2026-02-12T19:12:46.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:887: test_activate_osd: grep --quiet osd_max_backfills 2026-02-12T19:12:46.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:889: test_activate_osd: kill_daemons td/ceph-helpers TERM osd 2026-02-12T19:12:46.201 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:46.201 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:46.201 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:46.201 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:46.201 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:46.512 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:46.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:891: test_activate_osd: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-12T19:12:46.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-12T19:12:46.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T19:12:46.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-12T19:12:46.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T19:12:46.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:12:46.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:12:46.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:12:46.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:12:46.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:12:46.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:12:46.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:12:46.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T19:12:46.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T19:12:46.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:12:46.514 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T19:12:46.514 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:46.514 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:46.514 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:46.515 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:46.515 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:46.515 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:46.515 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:46.515 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:12:46.515 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:12:46.515 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:12:46.515 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:12:46.515 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:12:46.515 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:12:46.516 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:12:46.516 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T19:12:46.516 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-12T19:12:46.516 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:12:46.516 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:12:46.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-12T19:12:46.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-12T19:12:46.518 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-12T19:12:46.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-12T19:12:46.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T19:12:46.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T19:12:46.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T19:12:46.533 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:46.526+0000 7f27951fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:46.535 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:46.526+0000 7f27951fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:46.536 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:46.526+0000 7f27951fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:46.733 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-12T19:12:46.733 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:12:46.733 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:12:46.733 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:12:46.733 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:12:46.733 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:46.733 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:12:46.734 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:46.734 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:46.734 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:46.877 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:46.870+0000 7f27951fca40 -1 Falling back to public interface 2026-02-12T19:12:46.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:47.572 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:47.562+0000 7f27951fca40 -1 osd.0 5 log_to_monitors true 2026-02-12T19:12:47.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:47.947 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:47.947 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:12:47.948 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:12:47.948 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:47.948 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:48.154 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/2250453410,v1:127.0.0.1:6801/2250453410] [v2:127.0.0.1:6802/2250453410,v1:127.0.0.1:6803/2250453410] exists,up 82414629-1177-4cc7-842d-c9e867137252 2026-02-12T19:12:48.155 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:12:48.155 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:12:48.155 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:12:48.156 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: get_asok_path osd.0 2026-02-12T19:12:48.156 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:12:48.156 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:12:48.157 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:48.157 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:48.157 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:48.157 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-osd.0.asok 2026-02-12T19:12:48.157 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: CEPH_ARGS= 2026-02-12T19:12:48.157 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: ceph --format=json daemon /tmp/ceph-asok.52248/ceph-osd.0.asok config get osd_max_backfills 2026-02-12T19:12:48.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:893: test_activate_osd: local 'backfills={"osd_max_backfills":"20"}' 2026-02-12T19:12:48.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:894: test_activate_osd: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-12T19:12:48.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:896: test_activate_osd: teardown td/ceph-helpers 2026-02-12T19:12:48.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:12:48.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:12:48.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:12:48.216 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:48.216 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:48.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:48.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:48.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:48.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:48.524 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:12:48.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:12:48.526 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:12:48.527 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:12:48.527 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:12:48.528 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:12:48.529 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:48.529 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:12:48.529 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:12:48.530 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:48.531 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:12:48.532 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:12:48.532 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:12:48.535 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:12:48.535 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:48.536 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:48.536 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:12:48.538 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:12:48.538 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:12:48.538 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:12:48.538 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd_after_mark_down td/ceph-helpers 2026-02-12T19:12:48.538 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:900: test_activate_osd_after_mark_down: local dir=td/ceph-helpers 2026-02-12T19:12:48.538 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:902: test_activate_osd_after_mark_down: setup td/ceph-helpers 2026-02-12T19:12:48.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:12:48.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:12:48.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:12:48.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:12:48.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:12:48.539 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:48.539 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:48.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:48.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:48.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:48.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:48.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:12:48.541 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:12:48.541 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:12:48.542 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:12:48.542 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:12:48.543 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:12:48.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:48.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:12:48.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:12:48.544 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:48.545 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:12:48.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:12:48.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:12:48.548 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:12:48.548 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:48.548 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:48.548 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:12:48.549 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:12:48.549 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:12:48.549 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:12:48.550 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:12:48.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:48.551 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:48.551 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:12:48.552 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:12:48.553 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:12:48.553 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:12:48.553 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:12:48.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:904: test_activate_osd_after_mark_down: run_mon td/ceph-helpers a 2026-02-12T19:12:48.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:12:48.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:12:48.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:12:48.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:12:48.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:12:48.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-12T19:12:48.575 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:12:48.575 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:48.575 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:48.575 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:48.575 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:48.576 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:48.576 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:48.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:12:48.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:12:48.604 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:12:48.604 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:12:48.604 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:12:48.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:12:48.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:12:48.606 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:12:48.606 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:12:48.606 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:12:48.606 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:48.606 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:48.606 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:48.606 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:12:48.606 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:12:48.607 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:12:48.660 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:12:48.660 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:12:48.661 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:12:48.661 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:12:48.661 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:12:48.662 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:12:48.662 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:12:48.662 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:12:48.662 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:48.662 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:48.662 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:48.662 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:12:48.663 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:12:48.663 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:12:48.717 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:905: test_activate_osd_after_mark_down: run_mgr td/ceph-helpers x 2026-02-12T19:12:48.717 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:12:48.717 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:12:48.717 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:12:48.717 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:12:48.717 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:12:48.717 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:12:48.827 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:12:48.827 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:48.827 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:48.828 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:48.828 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:48.828 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:48.828 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:48.829 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:12:48.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:12:48.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:907: test_activate_osd_after_mark_down: run_osd td/ceph-helpers 0 2026-02-12T19:12:48.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:12:48.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:12:48.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:12:48.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:12:48.851 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:12:48.852 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:12:48.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:12:48.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:12:48.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:12:48.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:12:48.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:12:48.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:12:48.854 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:12:48.854 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f09fec1c-c915-41f5-aeea-6a981f83e458 2026-02-12T19:12:48.854 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 f09fec1c-c915-41f5-aeea-6a981f83e458' 2026-02-12T19:12:48.854 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 f09fec1c-c915-41f5-aeea-6a981f83e458 2026-02-12T19:12:48.854 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:12:48.863 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAwJo5py9M1MxAAWdPU0IJKPDwXzDKQBSfNHg== 2026-02-12T19:12:48.863 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAwJo5py9M1MxAAWdPU0IJKPDwXzDKQBSfNHg=="}' 2026-02-12T19:12:48.863 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f09fec1c-c915-41f5-aeea-6a981f83e458 -i td/ceph-helpers/0/new.json 2026-02-12T19:12:48.977 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:48.986 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:12:48.986 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAwJo5py9M1MxAAWdPU0IJKPDwXzDKQBSfNHg== --osd-uuid f09fec1c-c915-41f5-aeea-6a981f83e458 2026-02-12T19:12:49.003 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:48.994+0000 7f62c0c8ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:49.005 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:48.998+0000 7f62c0c8ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:49.006 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:48.998+0000 7f62c0c8ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:49.006 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:48.998+0000 7f62c0c8ba40 -1 bdev(0x5616c62c0400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:12:49.006 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:48.998+0000 7f62c0c8ba40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:12:49.747 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:12:49.748 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:12:49.749 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:12:49.749 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:12:49.749 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:12:49.869 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:12:49.869 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:12:49.869 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:12:49.870 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:12:49.870 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:12:49.870 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:12:49.893 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:49.886+0000 7f5236ab8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:49.896 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:49.890+0000 7f5236ab8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:49.897 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:49.890+0000 7f5236ab8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:49.977 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:12:49.977 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:12:49.977 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:12:49.977 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:12:49.977 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:12:49.977 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:49.978 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:49.978 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:12:49.978 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:49.978 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:50.086 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:50.401 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:50.394+0000 7f5236ab8a40 -1 Falling back to public interface 2026-02-12T19:12:51.087 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:51.088 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:51.088 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:12:51.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:12:51.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:51.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:51.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:51.229 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:51.222+0000 7f5236ab8a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:12:52.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:52.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:52.198 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:12:52.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:12:52.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:52.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:52.410 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3025702443,v1:127.0.0.1:6801/3025702443] [v2:127.0.0.1:6802/3025702443,v1:127.0.0.1:6803/3025702443] exists,up f09fec1c-c915-41f5-aeea-6a981f83e458 2026-02-12T19:12:52.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:12:52.411 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:12:52.411 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:12:52.412 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-12T19:12:52.412 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:12:52.412 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:12:52.412 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:52.412 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:52.412 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:52.413 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-osd.0.asok 2026-02-12T19:12:52.413 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-12T19:12:52.413 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.52248/ceph-osd.0.asok config get osd_max_backfills 2026-02-12T19:12:52.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:909: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"1"}' 2026-02-12T19:12:52.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: echo '{"osd_max_backfills":"1"}' 2026-02-12T19:12:52.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:910: test_activate_osd_after_mark_down: grep --quiet osd_max_backfills 2026-02-12T19:12:52.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:912: test_activate_osd_after_mark_down: kill_daemons td/ceph-helpers TERM osd 2026-02-12T19:12:52.473 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:52.473 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:52.473 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:52.473 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:52.473 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:52.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:52.781 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:913: test_activate_osd_after_mark_down: ceph osd down 0 2026-02-12T19:12:52.989 INFO:tasks.workunit.client.0.trial010.stderr:osd.0 is already down. 2026-02-12T19:12:52.998 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:914: test_activate_osd_after_mark_down: wait_for_osd down 0 2026-02-12T19:12:52.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-12T19:12:52.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:12:52.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:12:52.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:12:52.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:52.999 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:53.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:12:53.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:53.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-12T19:12:53.225 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/3025702443,v1:127.0.0.1:6801/3025702443] [v2:127.0.0.1:6802/3025702443,v1:127.0.0.1:6803/3025702443] exists f09fec1c-c915-41f5-aeea-6a981f83e458 2026-02-12T19:12:53.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:12:53.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:12:53.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:12:53.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:916: test_activate_osd_after_mark_down: activate_osd td/ceph-helpers 0 --osd-max-backfills 20 2026-02-12T19:12:53.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-12T19:12:53.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T19:12:53.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-12T19:12:53.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T19:12:53.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:12:53.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:12:53.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:12:53.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:12:53.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:12:53.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:12:53.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:12:53.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T19:12:53.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T19:12:53.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:12:53.230 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T19:12:53.230 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:12:53.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:12:53.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T19:12:53.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-max-backfills 20' 2026-02-12T19:12:53.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:12:53.232 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:12:53.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-12T19:12:53.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-max-backfills 20 2026-02-12T19:12:53.233 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-12T19:12:53.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-12T19:12:53.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T19:12:53.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T19:12:53.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T19:12:53.252 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:53.242+0000 7fc57885ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:53.253 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:53.246+0000 7fc57885ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:53.254 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:53.246+0000 7fc57885ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:53.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-12T19:12:53.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:12:53.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:12:53.444 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:12:53.445 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:12:53.445 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:53.445 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:53.446 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:12:53.446 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:53.446 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:53.653 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:53.725 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:53.718+0000 7fc57885ea40 -1 Falling back to public interface 2026-02-12T19:12:54.383 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:54.374+0000 7fc57885ea40 -1 osd.0 5 log_to_monitors true 2026-02-12T19:12:54.655 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:54.656 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:54.656 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:12:54.657 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:12:54.657 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:54.657 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:54.875 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/816096561,v1:127.0.0.1:6801/816096561] [v2:127.0.0.1:6802/816096561,v1:127.0.0.1:6803/816096561] exists,up f09fec1c-c915-41f5-aeea-6a981f83e458 2026-02-12T19:12:54.878 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:12:54.878 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:12:54.878 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:12:54.878 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: get_asok_path osd.0 2026-02-12T19:12:54.878 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:12:54.879 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:12:54.880 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:54.880 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:54.880 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:54.880 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-osd.0.asok 2026-02-12T19:12:54.880 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: CEPH_ARGS= 2026-02-12T19:12:54.880 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: ceph --format=json daemon /tmp/ceph-asok.52248/ceph-osd.0.asok config get osd_max_backfills 2026-02-12T19:12:54.937 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:918: test_activate_osd_after_mark_down: local 'backfills={"osd_max_backfills":"20"}' 2026-02-12T19:12:54.937 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:919: test_activate_osd_after_mark_down: test '{"osd_max_backfills":"20"}' = '{"osd_max_backfills":"20"}' 2026-02-12T19:12:54.938 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:921: test_activate_osd_after_mark_down: teardown td/ceph-helpers 2026-02-12T19:12:54.938 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:12:54.938 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:12:54.938 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:12:54.938 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:54.938 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:54.938 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:54.938 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:54.938 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:55.045 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:55.045 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:12:55.046 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:12:55.046 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:12:55.047 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:12:55.047 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:12:55.048 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:12:55.049 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:55.049 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:12:55.049 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:12:55.049 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:55.050 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:12:55.051 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:12:55.051 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:12:55.057 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:12:55.057 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:55.057 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:55.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:12:55.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:12:55.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:12:55.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:12:55.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_activate_osd_skip_benchmark td/ceph-helpers 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:925: test_activate_osd_skip_benchmark: local dir=td/ceph-helpers 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:927: test_activate_osd_skip_benchmark: setup td/ceph-helpers 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:55.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:55.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:55.062 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:12:55.062 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:12:55.062 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:12:55.064 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:12:55.064 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:12:55.064 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:12:55.065 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:55.065 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:12:55.066 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:12:55.066 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:12:55.067 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:12:55.068 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:12:55.068 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:12:55.069 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:12:55.069 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:55.069 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:55.069 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:12:55.070 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:12:55.070 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:12:55.070 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:12:55.072 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:12:55.072 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:55.072 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:55.072 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:12:55.073 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:12:55.074 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:12:55.074 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:12:55.074 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:12:55.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:929: test_activate_osd_skip_benchmark: run_mon td/ceph-helpers a 2026-02-12T19:12:55.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:12:55.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:12:55.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:12:55.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:12:55.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:12:55.075 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-12T19:12:55.096 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:12:55.096 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:55.096 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:55.096 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:55.096 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:55.096 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:55.096 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:55.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:12:55.122 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:12:55.126 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:12:55.126 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:12:55.126 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:12:55.126 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:12:55.127 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:12:55.127 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:12:55.127 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:12:55.127 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:12:55.127 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:55.127 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:55.128 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:55.128 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:12:55.128 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:12:55.128 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:12:55.178 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:12:55.179 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:12:55.179 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:12:55.179 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:12:55.179 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:12:55.180 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:12:55.180 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:12:55.180 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:12:55.180 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:55.180 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:55.180 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:55.180 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:12:55.180 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:12:55.180 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:12:55.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:930: test_activate_osd_skip_benchmark: run_mgr td/ceph-helpers x 2026-02-12T19:12:55.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:12:55.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:12:55.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:12:55.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:12:55.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:12:55.234 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:12:55.346 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:12:55.346 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:55.346 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:55.346 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:55.346 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:55.347 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:55.347 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:55.347 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:12:55.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:12:55.368 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:933: test_activate_osd_skip_benchmark: run_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-12T19:12:55.369 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:12:55.369 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:12:55.369 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:12:55.369 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:12:55.369 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:12:55.369 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:12:55.369 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:12:55.369 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:12:55.369 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:12:55.369 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:12:55.370 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:12:55.370 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:12:55.370 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:12:55.370 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:12:55.370 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:12:55.370 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:55.370 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:55.370 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:55.370 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:55.370 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:55.371 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:55.371 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:55.371 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:12:55.371 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:12:55.371 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:12:55.371 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:12:55.372 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:12:55.372 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:12:55.372 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:12:55.372 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:12:55.372 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:12:55.372 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:12:55.373 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-12T19:12:55.373 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:12:55.373 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:12:55.374 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e6b885ae-38d5-421a-a85d-f6c10bdec4b8 2026-02-12T19:12:55.374 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 e6b885ae-38d5-421a-a85d-f6c10bdec4b8' 2026-02-12T19:12:55.375 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 e6b885ae-38d5-421a-a85d-f6c10bdec4b8 2026-02-12T19:12:55.375 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:12:55.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA3Jo5ps8eIFhAAp5dKSCWLS5beg1sX92JDcw== 2026-02-12T19:12:55.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA3Jo5ps8eIFhAAp5dKSCWLS5beg1sX92JDcw=="}' 2026-02-12T19:12:55.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e6b885ae-38d5-421a-a85d-f6c10bdec4b8 -i td/ceph-helpers/0/new.json 2026-02-12T19:12:55.494 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:55.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:12:55.504 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true --mkfs --key AQA3Jo5ps8eIFhAAp5dKSCWLS5beg1sX92JDcw== --osd-uuid e6b885ae-38d5-421a-a85d-f6c10bdec4b8 2026-02-12T19:12:55.520 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:55.510+0000 7f1a2b479a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:55.521 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:55.514+0000 7f1a2b479a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:55.522 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:55.514+0000 7f1a2b479a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:55.522 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:55.514+0000 7f1a2b479a40 -1 bdev(0x5596e6c86400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:12:55.523 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:55.514+0000 7f1a2b479a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:12:56.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:12:56.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:12:56.348 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:12:56.349 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:12:56.349 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:12:56.465 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:12:56.466 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:12:56.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-12T19:12:56.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:12:56.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:12:56.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:12:56.491 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:56.482+0000 7f069c469a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:56.493 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:56.486+0000 7f069c469a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:56.494 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:56.486+0000 7f069c469a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:12:56.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:12:56.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:12:56.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:12:56.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:12:56.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:12:56.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:56.575 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:12:56.576 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:56.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:56.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:56.681 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:56.886 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:56.878+0000 7f069c469a40 -1 Falling back to public interface 2026-02-12T19:12:57.683 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:57.683 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:57.683 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:12:57.684 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:12:57.684 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:57.684 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:57.874 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:12:57.866+0000 7f069c469a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:12:57.904 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:12:58.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:12:58.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:58.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:12:58.906 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:12:58.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:58.906 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:12:59.119 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3130641619,v1:127.0.0.1:6801/3130641619] [v2:127.0.0.1:6802/3130641619,v1:127.0.0.1:6803/3130641619] exists,up e6b885ae-38d5-421a-a85d-f6c10bdec4b8 2026-02-12T19:12:59.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:12:59.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:12:59.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:12:59.122 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-12T19:12:59.122 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:12:59.122 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:12:59.122 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:59.122 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:59.123 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:59.123 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-osd.0.asok 2026-02-12T19:12:59.123 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-12T19:12:59.123 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.52248/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-12T19:12:59.179 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:936: test_activate_osd_skip_benchmark: local 'max_iops_hdd_def={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-12T19:12:59.180 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-12T19:12:59.180 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:12:59.180 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:12:59.181 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:12:59.181 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:59.181 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:59.181 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-osd.0.asok 2026-02-12T19:12:59.181 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-12T19:12:59.181 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.52248/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-12T19:12:59.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:938: test_activate_osd_skip_benchmark: local 'max_iops_ssd_def={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-12T19:12:59.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:940: test_activate_osd_skip_benchmark: kill_daemons td/ceph-helpers TERM osd 2026-02-12T19:12:59.243 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:12:59.243 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:12:59.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:12:59.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:12:59.244 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:12:59.552 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:12:59.552 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:941: test_activate_osd_skip_benchmark: ceph osd down 0 2026-02-12T19:12:59.761 INFO:tasks.workunit.client.0.trial010.stderr:osd.0 is already down. 2026-02-12T19:12:59.771 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:942: test_activate_osd_skip_benchmark: wait_for_osd down 0 2026-02-12T19:12:59.771 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=down 2026-02-12T19:12:59.772 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:12:59.772 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:12:59.772 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:12:59.772 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:12:59.772 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:12:59.773 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:12:59.773 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:12:59.773 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 down' 2026-02-12T19:12:59.986 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 down in weight 1 up_from 5 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/3130641619,v1:127.0.0.1:6801/3130641619] [v2:127.0.0.1:6802/3130641619,v1:127.0.0.1:6803/3130641619] exists e6b885ae-38d5-421a-a85d-f6c10bdec4b8 2026-02-12T19:12:59.989 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:12:59.989 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:12:59.989 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:12:59.989 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:946: test_activate_osd_skip_benchmark: activate_osd td/ceph-helpers 0 --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-12T19:12:59.989 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-12T19:12:59.989 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T19:12:59.989 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-12T19:12:59.989 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T19:12:59.990 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:12:59.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:12:59.992 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:12:59.992 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T19:12:59.992 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+='--osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true' 2026-02-12T19:12:59.992 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:12:59.992 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:12:59.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-12T19:12:59.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd-op-queue=mclock_scheduler --osd-mclock-skip-benchmark=true 2026-02-12T19:12:59.993 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-12T19:12:59.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-12T19:12:59.994 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T19:12:59.994 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T19:12:59.994 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T19:13:00.013 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:00.006+0000 7fc9d17b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:00.015 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:00.006+0000 7fc9d17b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:00.016 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:00.006+0000 7fc9d17b1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:00.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-12T19:13:00.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:13:00.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:13:00.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:13:00.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:13:00.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:00.200 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:00.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:13:00.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:00.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:00.350 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:00.342+0000 7fc9d17b1a40 -1 Falling back to public interface 2026-02-12T19:13:00.412 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:00.952 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:00.946+0000 7fc9d17b1a40 -1 osd.0 5 log_to_monitors true 2026-02-12T19:13:01.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:01.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:01.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:13:01.414 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:13:01.415 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:01.415 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:01.633 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 8 up_thru 0 down_at 6 last_clean_interval [0,0) [v2:127.0.0.1:6800/2718978403,v1:127.0.0.1:6801/2718978403] [v2:127.0.0.1:6802/2718978403,v1:127.0.0.1:6803/2718978403] exists,up e6b885ae-38d5-421a-a85d-f6c10bdec4b8 2026-02-12T19:13:01.634 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:13:01.634 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:13:01.634 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:13:01.635 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-12T19:13:01.636 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:13:01.636 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:13:01.636 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:01.636 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:01.636 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:01.636 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-osd.0.asok 2026-02-12T19:13:01.636 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-12T19:13:01.636 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.52248/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_hdd 2026-02-12T19:13:01.695 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:949: test_activate_osd_skip_benchmark: local 'max_iops_hdd_after_boot={"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-12T19:13:01.696 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: get_asok_path osd.0 2026-02-12T19:13:01.696 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:13:01.696 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:13:01.697 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:01.697 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:01.697 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:01.697 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-osd.0.asok 2026-02-12T19:13:01.697 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: CEPH_ARGS= 2026-02-12T19:13:01.698 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: ceph --format=json daemon /tmp/ceph-asok.52248/ceph-osd.0.asok config get osd_mclock_max_capacity_iops_ssd 2026-02-12T19:13:01.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:951: test_activate_osd_skip_benchmark: local 'max_iops_ssd_after_boot={"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-12T19:13:01.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:953: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' = '{"osd_mclock_max_capacity_iops_hdd":"315.000000"}' 2026-02-12T19:13:01.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:954: test_activate_osd_skip_benchmark: test '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' = '{"osd_mclock_max_capacity_iops_ssd":"21500.000000"}' 2026-02-12T19:13:01.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:956: test_activate_osd_skip_benchmark: teardown td/ceph-helpers 2026-02-12T19:13:01.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:01.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:01.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:01.758 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:01.758 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:01.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:01.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:01.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:01.865 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:01.866 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:01.866 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:01.867 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:01.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:01.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:01.869 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:01.869 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:01.869 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:01.870 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:01.870 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:01.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:01.872 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:01.872 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:01.877 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:01.877 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:01.877 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:01.877 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:01.880 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:01.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:01.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:01.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_destroy_osd td/ceph-helpers 2026-02-12T19:13:01.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:787: test_destroy_osd: local dir=td/ceph-helpers 2026-02-12T19:13:01.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:789: test_destroy_osd: setup td/ceph-helpers 2026-02-12T19:13:01.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:01.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:01.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:01.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:01.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:01.882 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:01.882 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:01.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:01.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:01.882 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:01.883 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:01.883 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:01.883 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:01.883 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:01.884 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:01.884 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:01.884 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:01.885 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:01.885 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:01.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:01.886 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:01.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:01.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:01.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:01.889 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:01.889 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:01.890 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:01.890 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:01.891 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:01.891 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:01.891 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:01.892 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:01.892 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:01.892 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:01.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:01.894 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:01.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:01.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:01.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:01.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:790: test_destroy_osd: run_mon td/ceph-helpers a 2026-02-12T19:13:01.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:13:01.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:13:01.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:13:01.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:13:01.896 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:13:01.896 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-12T19:13:01.917 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:13:01.917 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:01.917 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:01.918 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:01.918 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:01.918 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:01.918 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:01.918 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:13:01.943 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:13:01.944 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:13:01.944 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:01.945 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:01.945 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:13:01.946 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:13:01.946 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:01.946 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:01.946 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:01.946 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:01.946 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:01.946 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:01.946 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:01.947 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:01.947 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:13:02.002 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:13:02.003 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:02.003 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:02.003 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:13:02.003 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:13:02.004 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:02.004 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:02.004 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:02.005 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:02.005 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:02.005 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:02.005 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:02.005 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:02.035 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:13:02.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:791: test_destroy_osd: run_mgr td/ceph-helpers x 2026-02-12T19:13:02.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:13:02.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:13:02.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:13:02.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:13:02.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:13:02.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:13:02.176 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:13:02.176 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:02.176 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:02.176 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:02.176 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:02.176 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:02.177 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:02.177 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:02.178 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:02.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:792: test_destroy_osd: run_osd td/ceph-helpers 0 2026-02-12T19:13:02.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:13:02.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:13:02.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:13:02.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:13:02.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:13:02.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:13:02.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:13:02.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:02.199 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:13:02.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:13:02.201 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:13:02.201 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:13:02.201 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=108cfd6c-f1d7-4f84-ba80-64760d76982f 2026-02-12T19:13:02.201 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 108cfd6c-f1d7-4f84-ba80-64760d76982f 2026-02-12T19:13:02.202 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 108cfd6c-f1d7-4f84-ba80-64760d76982f' 2026-02-12T19:13:02.202 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:13:02.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA+Jo5pKwSiDBAAQlLH0Fi6P/agAYD+a95Tow== 2026-02-12T19:13:02.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA+Jo5pKwSiDBAAQlLH0Fi6P/agAYD+a95Tow=="}' 2026-02-12T19:13:02.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 108cfd6c-f1d7-4f84-ba80-64760d76982f -i td/ceph-helpers/0/new.json 2026-02-12T19:13:02.330 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:02.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:13:02.339 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA+Jo5pKwSiDBAAQlLH0Fi6P/agAYD+a95Tow== --osd-uuid 108cfd6c-f1d7-4f84-ba80-64760d76982f 2026-02-12T19:13:02.355 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:02.346+0000 7f29b058ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:02.356 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:02.350+0000 7f29b058ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:02.358 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:02.350+0000 7f29b058ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:02.358 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:02.350+0000 7f29b058ea40 -1 bdev(0x55e942016400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:13:02.358 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:02.350+0000 7f29b058ea40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:13:03.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:13:03.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:13:03.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:13:03.061 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:13:03.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:13:03.180 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:13:03.181 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:13:03.182 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:13:03.182 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:13:03.182 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:13:03.182 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:13:03.205 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:03.198+0000 7f951fd51a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:03.207 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:03.198+0000 7f951fd51a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:03.208 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:03.202+0000 7f951fd51a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:03.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:13:03.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:13:03.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:13:03.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:13:03.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:13:03.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:03.292 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:03.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:13:03.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:03.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:03.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:03.754 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:03.746+0000 7f951fd51a40 -1 Falling back to public interface 2026-02-12T19:13:04.398 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:04.399 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:04.399 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:13:04.399 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:13:04.400 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:04.400 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:04.507 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:04.790 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:04.782+0000 7f951fd51a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:13:05.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:05.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:05.509 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:13:05.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:13:05.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:05.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:05.727 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2123280535,v1:127.0.0.1:6801/2123280535] [v2:127.0.0.1:6802/2123280535,v1:127.0.0.1:6803/2123280535] exists,up 108cfd6c-f1d7-4f84-ba80-64760d76982f 2026-02-12T19:13:05.728 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:13:05.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:13:05.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:13:05.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:793: test_destroy_osd: destroy_osd td/ceph-helpers 0 2026-02-12T19:13:05.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:775: destroy_osd: local dir=td/ceph-helpers 2026-02-12T19:13:05.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:776: destroy_osd: local id=0 2026-02-12T19:13:05.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:778: destroy_osd: ceph osd out osd.0 2026-02-12T19:13:05.970 INFO:tasks.workunit.client.0.trial010.stderr:osd.0 is already out. 2026-02-12T19:13:05.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:779: destroy_osd: kill_daemons td/ceph-helpers TERM osd.0 2026-02-12T19:13:05.980 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:05.980 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:05.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:05.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:05.981 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:06.288 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:06.289 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:780: destroy_osd: ceph osd down osd.0 2026-02-12T19:13:06.501 INFO:tasks.workunit.client.0.trial010.stderr:osd.0 is already down. 2026-02-12T19:13:06.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:781: destroy_osd: ceph osd purge osd.0 --yes-i-really-mean-it 2026-02-12T19:13:06.732 INFO:tasks.workunit.client.0.trial010.stderr:osd.0 does not exist 2026-02-12T19:13:06.742 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:782: destroy_osd: teardown td/ceph-helpers/0 2026-02-12T19:13:06.742 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers/0 2026-02-12T19:13:06.743 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:06.743 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers/0 KILL 2026-02-12T19:13:06.743 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:06.743 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:06.743 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:06.743 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:06.743 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:06.745 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:06.745 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:06.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:06.747 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:06.748 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:06.748 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:06.749 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:06.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:06.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:06.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:06.750 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:06.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:06.752 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:06.752 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers/0 2026-02-12T19:13:06.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:06.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:06.755 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:06.755 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:06.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:06.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:06.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:783: destroy_osd: rm -fr td/ceph-helpers/0 2026-02-12T19:13:06.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: ceph osd dump 2026-02-12T19:13:06.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:794: test_destroy_osd: grep 'osd. ' 2026-02-12T19:13:06.971 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:795: test_destroy_osd: teardown td/ceph-helpers 2026-02-12T19:13:06.971 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:06.971 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:06.971 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:06.971 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:06.971 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:06.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:06.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:06.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:07.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:07.078 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:07.079 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:07.080 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:07.081 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:07.081 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:07.081 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:07.082 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:07.083 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:07.083 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:07.083 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:07.084 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:07.085 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:07.085 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:07.089 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:07.089 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.090 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:07.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:07.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:07.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:07.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_display_logs td/ceph-helpers 2026-02-12T19:13:07.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2158: test_display_logs: local dir=td/ceph-helpers 2026-02-12T19:13:07.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2160: test_display_logs: setup td/ceph-helpers 2026-02-12T19:13:07.093 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:07.093 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:07.093 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:07.093 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:07.093 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:07.093 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:07.093 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:07.093 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:07.093 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:07.093 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:07.094 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:07.094 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:07.095 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:07.095 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:07.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:07.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:07.097 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:07.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:07.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:07.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:07.099 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:07.099 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:07.100 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:07.101 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:07.102 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:07.102 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.102 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.102 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:07.103 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:07.103 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:07.103 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:07.105 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:07.105 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.105 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.105 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:07.106 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:07.107 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:07.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:07.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:07.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2161: test_display_logs: run_mon td/ceph-helpers a 2026-02-12T19:13:07.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:13:07.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:13:07.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:13:07.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:13:07.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:13:07.108 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-12T19:13:07.129 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:13:07.129 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:07.129 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:07.129 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:07.129 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.129 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.130 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:07.130 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:13:07.159 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:13:07.160 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:13:07.160 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:07.160 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:07.161 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:13:07.161 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:13:07.162 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:07.162 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:07.162 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:07.162 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:07.162 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.162 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.162 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:07.162 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:07.162 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:13:07.216 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:13:07.216 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:07.216 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:07.216 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:13:07.217 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:13:07.218 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:07.218 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:07.218 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:07.218 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:07.218 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.218 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.218 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:07.218 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:07.218 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:13:07.271 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2162: test_display_logs: kill_daemons td/ceph-helpers 2026-02-12T19:13:07.272 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:07.272 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:07.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:07.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:07.273 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:07.379 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:07.379 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2163: test_display_logs: display_logs td/ceph-helpers 2026-02-12T19:13:07.379 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2148: display_logs: local dir=td/ceph-helpers 2026-02-12T19:13:07.379 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2150: display_logs: find td/ceph-helpers -maxdepth 1 -name '*.log' 2026-02-12T19:13:07.379 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-12T19:13:07.380 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2152: display_logs: echo '======================= td/ceph-helpers/mon.a.log' 2026-02-12T19:13:07.380 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2153: display_logs: cat td/ceph-helpers/mon.a.log 2026-02-12T19:13:07.380 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2151: display_logs: read file 2026-02-12T19:13:07.381 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2164: test_display_logs: grep --quiet mon.a.log td/ceph-helpers/log.out 2026-02-12T19:13:07.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2165: test_display_logs: teardown td/ceph-helpers 2026-02-12T19:13:07.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:07.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:07.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:07.383 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:07.383 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:07.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:07.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:07.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:07.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:07.385 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:07.386 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:07.386 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:07.387 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:07.387 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:07.387 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:07.388 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:07.389 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:07.389 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:07.389 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:07.390 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:07.391 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:07.391 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:07.393 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:07.393 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.393 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.393 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:07.395 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:07.395 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_erasure_code_plugin_exists td/ceph-helpers 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2129: test_erasure_code_plugin_exists: local dir=td/ceph-helpers 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2131: test_erasure_code_plugin_exists: setup td/ceph-helpers 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:07.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:07.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:07.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:07.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:07.397 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:07.398 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:07.398 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:07.399 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:07.399 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:07.399 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:07.400 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:07.400 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:07.401 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:07.401 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:07.402 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:07.403 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:07.403 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:07.404 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:07.404 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.404 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.405 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:07.406 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:07.406 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:07.406 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:07.407 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:07.407 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.407 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.408 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:07.409 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:07.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:07.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:07.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:07.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2132: test_erasure_code_plugin_exists: run_mon td/ceph-helpers a 2026-02-12T19:13:07.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:13:07.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:13:07.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:13:07.411 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:13:07.411 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:13:07.411 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-12T19:13:07.432 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:13:07.432 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:07.432 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:07.432 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:07.432 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.432 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.432 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:07.433 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:13:07.461 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:13:07.462 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:13:07.462 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:07.462 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:07.462 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:13:07.464 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:13:07.464 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:07.464 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:07.464 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:07.464 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:07.464 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.464 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.464 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:07.464 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:07.464 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:13:07.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:13:07.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:07.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:07.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:13:07.522 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:13:07.522 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:07.522 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:07.523 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:07.523 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:07.523 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.523 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.523 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:07.523 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:07.523 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:13:07.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2133: test_erasure_code_plugin_exists: run_mgr td/ceph-helpers x 2026-02-12T19:13:07.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:13:07.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:13:07.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:13:07.578 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:13:07.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:13:07.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:13:07.694 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:13:07.694 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:07.694 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:07.694 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:07.694 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:07.694 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:07.694 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:07.695 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:07.696 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:07.714 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2134: test_erasure_code_plugin_exists: erasure_code_plugin_exists jerasure 2026-02-12T19:13:07.714 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=jerasure 2026-02-12T19:13:07.714 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-12T19:13:07.714 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-12T19:13:07.715 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-12T19:13:07.715 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-12T19:13:07.715 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-12T19:13:07.715 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='jerasure.*No such file' 2026-02-12T19:13:07.715 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=jerasure 2026-02-12T19:13:07.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-12T19:13:07.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-12T19:13:07.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-12T19:13:07.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-12T19:13:07.990 INFO:tasks.workunit.client.0.trial010.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-12T19:13:07.998 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-12T19:13:07.998 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2135: test_erasure_code_plugin_exists: erasure_code_plugin_exists FAKE 2026-02-12T19:13:07.998 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=FAKE 2026-02-12T19:13:07.998 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-12T19:13:07.998 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-12T19:13:07.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-12T19:13:07.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-12T19:13:07.999 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-12T19:13:07.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='FAKE.*No such file' 2026-02-12T19:13:08.000 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=FAKE 2026-02-12T19:13:08.095 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s='Error EIO: load dlopen(/usr/lib/ceph/erasure-code/libec_FAKE.so): /usr/lib/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory' 2026-02-12T19:13:08.095 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=5 2026-02-12T19:13:08.095 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 5 -eq 0 ']' 2026-02-12T19:13:08.095 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: echo Error EIO: load 'dlopen(/usr/lib/ceph/erasure-code/libec_FAKE.so):' /usr/lib/ceph/erasure-code/libec_FAKE.so: cannot open shared object file: No such file or directory 2026-02-12T19:13:08.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2117: erasure_code_plugin_exists: grep --quiet 'FAKE.*No such file' 2026-02-12T19:13:08.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2119: erasure_code_plugin_exists: status=1 2026-02-12T19:13:08.097 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 1 2026-02-12T19:13:08.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2136: test_erasure_code_plugin_exists: teardown td/ceph-helpers 2026-02-12T19:13:08.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:08.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:08.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:08.098 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:08.098 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:08.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:08.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:08.098 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:08.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:08.204 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:08.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:08.205 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:08.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:08.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:08.207 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:08.208 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.208 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:08.208 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:08.208 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.209 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:08.210 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:08.210 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:08.212 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:08.212 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.212 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.212 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:08.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:08.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:08.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:08.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_expect_failure td/ceph-helpers 2026-02-12T19:13:08.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: test_expect_failure: local dir=td/ceph-helpers 2026-02-12T19:13:08.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2039: test_expect_failure: setup td/ceph-helpers 2026-02-12T19:13:08.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:08.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:08.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:08.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:08.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:08.216 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:08.216 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:08.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:08.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:08.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:08.217 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:08.217 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:08.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:08.218 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:08.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:08.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:08.219 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:08.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:08.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:08.221 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.221 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:08.222 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:08.222 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:08.224 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:08.224 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.224 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.224 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:08.225 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:08.225 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:08.225 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:08.226 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:08.226 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.226 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.226 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:08.228 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:08.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:08.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:08.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:08.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2040: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 1' 2026-02-12T19:13:08.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-12T19:13:08.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-12T19:13:08.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-12T19:13:08.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-12T19:13:08.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-12T19:13:08.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 1' 2026-02-12T19:13:08.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-12T19:13:08.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-12T19:13:08.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-12T19:13:08.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-12T19:13:08.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2042: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo FAIL ; exit 0' 2026-02-12T19:13:08.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-12T19:13:08.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-12T19:13:08.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-12T19:13:08.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-12T19:13:08.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-12T19:13:08.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo FAIL ; exit 0' 2026-02-12T19:13:08.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2023: expect_failure: success=true 2026-02-12T19:13:08.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: true 2026-02-12T19:13:08.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-12T19:13:08.234 INFO:tasks.workunit.client.0.trial010.stderr:FAIL 2026-02-12T19:13:08.234 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-12T19:13:08.234 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2043: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-12T19:13:08.236 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2045: test_expect_failure: expect_failure td/ceph-helpers FAIL bash -c 'echo UNEXPECTED ; exit 1' 2026-02-12T19:13:08.236 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/ceph-helpers 2026-02-12T19:13:08.236 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-12T19:13:08.236 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=FAIL 2026-02-12T19:13:08.236 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-12T19:13:08.236 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-12T19:13:08.236 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: bash -c 'echo UNEXPECTED ; exit 1' 2026-02-12T19:13:08.237 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-12T19:13:08.237 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-12T19:13:08.237 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-12T19:13:08.238 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: cat td/ceph-helpers/out 2026-02-12T19:13:08.240 INFO:tasks.workunit.client.0.trial010.stderr:UNEXPECTED 2026-02-12T19:13:08.240 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2030: expect_failure: return 1 2026-02-12T19:13:08.240 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2046: test_expect_failure: grep --quiet FAIL td/ceph-helpers/out 2026-02-12T19:13:08.241 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2047: test_expect_failure: teardown td/ceph-helpers 2026-02-12T19:13:08.241 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:08.242 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:08.242 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:08.242 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:08.242 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:08.242 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:08.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:08.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:08.244 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:08.244 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:08.245 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:08.246 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:08.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:08.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:08.247 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:08.248 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.248 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:08.249 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:08.249 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:08.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:08.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:08.252 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:08.252 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.252 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:08.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:08.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:08.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_file_with_random_data td/ceph-helpers 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2350: test_file_with_random_data: local dir=td/ceph-helpers 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2352: test_file_with_random_data: setup td/ceph-helpers 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:08.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:08.256 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:08.256 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:08.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:08.258 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:08.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:08.259 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:08.259 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:08.260 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.260 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:08.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:08.261 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.262 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:08.262 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:08.263 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:08.264 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:08.264 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.264 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.264 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:08.265 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:08.265 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:08.265 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:08.266 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:08.266 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.266 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.267 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:08.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:08.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:08.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:08.269 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:08.269 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2353: test_file_with_random_data: file_with_random_data 4000 2026-02-12T19:13:08.269 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2343: file_with_random_data: local size_bytes=4000 2026-02-12T19:13:08.270 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: mktemp 2026-02-12T19:13:08.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local file=/tmp/tmp.vXzIJdmf7q 2026-02-12T19:13:08.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.vXzIJdmf7q bs=4000 count=1 2026-02-12T19:13:08.272 INFO:tasks.workunit.client.0.trial010.stderr:1+0 records in 2026-02-12T19:13:08.272 INFO:tasks.workunit.client.0.trial010.stderr:1+0 records out 2026-02-12T19:13:08.272 INFO:tasks.workunit.client.0.trial010.stderr:4000 bytes (4.0 kB, 3.9 KiB) copied, 9.3052e-05 s, 43.0 MB/s 2026-02-12T19:13:08.272 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: printf %s /tmp/tmp.vXzIJdmf7q 2026-02-12T19:13:08.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2353: test_file_with_random_data: local file=/tmp/tmp.vXzIJdmf7q 2026-02-12T19:13:08.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2354: test_file_with_random_data: test -f /tmp/tmp.vXzIJdmf7q 2026-02-12T19:13:08.273 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2355: test_file_with_random_data: stat -c %s /tmp/tmp.vXzIJdmf7q 2026-02-12T19:13:08.273 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2355: test_file_with_random_data: test 4000 = 4000 2026-02-12T19:13:08.274 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2356: test_file_with_random_data: rm /tmp/tmp.vXzIJdmf7q 2026-02-12T19:13:08.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2357: test_file_with_random_data: teardown td/ceph-helpers 2026-02-12T19:13:08.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:08.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:08.275 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:08.275 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:08.275 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:08.276 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:08.276 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:08.276 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:08.277 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:08.278 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:08.279 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:08.279 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:08.280 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:08.281 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:08.281 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:08.282 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.282 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:08.282 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:08.283 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.283 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:08.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:08.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:08.286 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:08.286 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.286 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:08.289 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:08.289 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:08.289 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:08.289 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_flush_pg_stats td/ceph-helpers 2026-02-12T19:13:08.289 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2280: test_flush_pg_stats: local dir=td/ceph-helpers 2026-02-12T19:13:08.289 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2282: test_flush_pg_stats: setup td/ceph-helpers 2026-02-12T19:13:08.289 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:08.289 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:08.289 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:08.290 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:08.290 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:08.290 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:08.290 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:08.290 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:08.290 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:08.290 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:08.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:08.291 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:08.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:08.292 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:08.293 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:08.293 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:08.294 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:08.295 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.295 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:08.295 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:08.296 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:08.296 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:08.298 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:08.298 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:08.299 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:08.299 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.299 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.299 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:08.300 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:08.300 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:08.300 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:08.301 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:08.301 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.301 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:08.303 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:08.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:08.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:08.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:08.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2283: test_flush_pg_stats: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:08.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:13:08.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:13:08.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:13:08.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:13:08.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:13:08.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:08.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:13:08.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:08.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:08.326 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:08.326 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.326 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.327 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:08.327 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:08.356 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:13:08.357 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:13:08.357 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:08.358 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:08.358 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:13:08.358 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:13:08.359 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:08.359 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:08.359 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:08.359 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:08.359 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.359 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.359 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:08.360 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:08.360 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:13:08.415 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:13:08.415 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:08.415 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:08.415 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:13:08.416 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:13:08.417 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:08.417 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:08.417 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:08.417 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:08.417 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.417 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.417 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:08.417 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:08.417 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:13:08.471 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2284: test_flush_pg_stats: run_mgr td/ceph-helpers x 2026-02-12T19:13:08.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:13:08.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:13:08.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:13:08.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:13:08.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:13:08.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:13:08.584 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:13:08.584 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:08.584 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:08.585 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:08.585 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.585 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.585 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:08.586 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:08.587 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:08.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2285: test_flush_pg_stats: run_osd td/ceph-helpers 0 2026-02-12T19:13:08.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:13:08.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:13:08.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:13:08.608 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:13:08.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:13:08.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:13:08.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:13:08.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:13:08.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:13:08.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:13:08.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:13:08.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:13:08.611 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:13:08.611 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=41cfa497-4c8d-44ad-ada5-44b27037aa03 2026-02-12T19:13:08.611 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 41cfa497-4c8d-44ad-ada5-44b27037aa03' 2026-02-12T19:13:08.611 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 41cfa497-4c8d-44ad-ada5-44b27037aa03 2026-02-12T19:13:08.611 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:13:08.622 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBEJo5pUeTRJBAAdUofmf99CuMkZCJXhhcfyA== 2026-02-12T19:13:08.622 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBEJo5pUeTRJBAAdUofmf99CuMkZCJXhhcfyA=="}' 2026-02-12T19:13:08.622 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 41cfa497-4c8d-44ad-ada5-44b27037aa03 -i td/ceph-helpers/0/new.json 2026-02-12T19:13:08.733 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:08.741 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:13:08.742 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBEJo5pUeTRJBAAdUofmf99CuMkZCJXhhcfyA== --osd-uuid 41cfa497-4c8d-44ad-ada5-44b27037aa03 2026-02-12T19:13:08.757 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:08.749+0000 7fb9719e6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:08.759 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:08.749+0000 7fb9719e6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:08.759 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:08.749+0000 7fb9719e6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:08.760 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:08.749+0000 7fb9719e6a40 -1 bdev(0x557a8293a400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:13:08.760 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:08.749+0000 7fb9719e6a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:13:09.527 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:13:09.527 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:13:09.528 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:13:09.528 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:13:09.529 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:13:09.643 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:13:09.643 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:13:09.644 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:13:09.644 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:13:09.644 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:13:09.644 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:13:09.669 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:09.661+0000 7f8733a9aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:09.671 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:09.661+0000 7f8733a9aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:09.672 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:09.665+0000 7f8733a9aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:09.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:13:09.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:13:09.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:13:09.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:13:09.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:13:09.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:09.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:13:09.756 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:09.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:09.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:09.869 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:10.210 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:10.201+0000 7f8733a9aa40 -1 Falling back to public interface 2026-02-12T19:13:10.870 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:10.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:10.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:13:10.871 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:13:10.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:10.872 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:10.978 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:11.146 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:11.137+0000 7f8733a9aa40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:13:11.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:11.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:11.980 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:13:11.981 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:13:11.981 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:11.981 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:12.199 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1116502121,v1:127.0.0.1:6801/1116502121] [v2:127.0.0.1:6802/1116502121,v1:127.0.0.1:6803/1116502121] exists,up 41cfa497-4c8d-44ad-ada5-44b27037aa03 2026-02-12T19:13:12.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:13:12.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:13:12.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:13:12.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2286: test_flush_pg_stats: create_rbd_pool 2026-02-12T19:13:12.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:13:12.404 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:13:12.415 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:13:12.415 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:13:12.644 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:13:12.654 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:13:13.655 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:13:13.968 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2287: test_flush_pg_stats: rados -p rbd put obj /etc/group 2026-02-12T19:13:13.987 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2288: test_flush_pg_stats: flush_pg_stats 2026-02-12T19:13:13.987 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:13:13.987 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:13:14.210 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:13:14.210 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:13:14.210 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:13:14.210 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:13:14.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-12T19:13:14.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-12T19:13:14.293 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-12T19:13:14.293 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:13:14.293 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-12T19:13:14.293 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:13:14.293 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:13:14.294 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-12T19:13:14.294 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:13:14.296 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-12T19:13:14.296 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-12T19:13:14.296 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836484 2026-02-12T19:13:14.296 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:13:14.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-12T19:13:14.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:13:15.527 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:13:15.528 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:13:15.737 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-12T19:13:15.737 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2289: test_flush_pg_stats: local 'jq_filter=.pools | .[] | select(.name == "rbd") | .stats' 2026-02-12T19:13:15.737 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: ceph df detail --format=json 2026-02-12T19:13:15.738 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored' 2026-02-12T19:13:16.018 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2290: test_flush_pg_stats: stored=8348 2026-02-12T19:13:16.019 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: ceph df detail --format=json 2026-02-12T19:13:16.019 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: jq '.pools | .[] | select(.name == "rbd") | .stats.stored_raw' 2026-02-12T19:13:16.293 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2291: test_flush_pg_stats: stored_raw=8348 2026-02-12T19:13:16.293 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2292: test_flush_pg_stats: test 8348 -gt 0 2026-02-12T19:13:16.293 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2293: test_flush_pg_stats: test 8348 == 8348 2026-02-12T19:13:16.294 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2294: test_flush_pg_stats: teardown td/ceph-helpers 2026-02-12T19:13:16.294 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:16.294 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:16.294 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:16.294 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:16.294 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:16.294 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:16.294 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:16.294 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:16.401 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:16.402 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:16.402 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:16.403 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:16.404 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:16.404 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:16.405 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:16.406 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:16.406 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:16.406 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:16.407 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:16.407 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:16.409 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:16.409 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:16.414 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:16.414 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:16.414 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:16.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:16.416 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:16.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:16.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:16.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_config td/ceph-helpers 2026-02-12T19:13:16.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: test_get_config: local dir=td/ceph-helpers 2026-02-12T19:13:16.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1130: test_get_config: setup td/ceph-helpers 2026-02-12T19:13:16.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:16.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:16.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:16.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:16.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:16.417 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:16.418 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:16.418 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:16.418 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:16.418 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:16.418 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:16.419 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:16.420 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:16.420 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:16.421 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:16.421 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:16.421 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:16.422 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:16.422 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:16.423 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:16.423 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:16.424 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:16.425 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:16.425 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:16.426 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:16.426 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:16.427 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:16.427 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:16.428 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:16.428 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:16.428 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:16.429 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:16.429 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:16.429 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:16.429 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:16.431 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:16.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:16.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:16.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:16.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1131: test_get_config: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:16.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:13:16.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:13:16.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:13:16.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:13:16.433 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:13:16.433 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:16.453 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:13:16.453 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:16.453 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:16.453 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:16.453 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:16.453 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:16.454 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:16.454 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:16.485 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:13:16.486 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:13:16.486 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:16.486 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:16.486 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:13:16.487 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:13:16.488 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:16.488 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:16.488 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:16.488 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:16.488 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:16.488 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:16.488 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:16.488 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:16.488 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:13:16.543 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:13:16.544 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:16.544 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:16.544 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:13:16.544 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:13:16.545 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:16.545 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:16.545 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:16.545 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:16.545 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:16.546 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:16.546 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:16.546 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:16.546 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:13:16.602 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: get_config mon a osd_pool_default_size 2026-02-12T19:13:16.602 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:16.602 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:16.602 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_pool_default_size 2026-02-12T19:13:16.602 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_pool_default_size 2026-02-12T19:13:16.603 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:16.603 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:16.603 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:16.604 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:16.604 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:16.604 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:16.604 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:16.604 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:16.604 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get osd_pool_default_size 2026-02-12T19:13:16.660 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: test_get_config: test 1 = 1 2026-02-12T19:13:16.660 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1133: test_get_config: run_mgr td/ceph-helpers x 2026-02-12T19:13:16.660 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:13:16.660 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:13:16.661 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:13:16.661 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:13:16.661 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:13:16.661 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:13:16.777 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:13:16.778 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:16.778 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:16.778 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:16.778 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:16.778 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:16.778 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:16.778 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:16.779 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:16.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1134: test_get_config: run_osd td/ceph-helpers 0 --osd_max_scrubs=3 2026-02-12T19:13:16.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:13:16.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:13:16.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:13:16.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:13:16.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:13:16.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:13:16.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:13:16.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:13:16.801 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:13:16.801 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:13:16.801 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:13:16.801 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:13:16.801 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:13:16.801 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:13:16.801 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:13:16.801 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:16.801 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:16.801 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:16.801 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:13:16.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_max_scrubs=3 2026-02-12T19:13:16.803 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:13:16.803 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:13:16.803 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0e938e1a-420c-4087-b8d1-93457eeb20e4 2026-02-12T19:13:16.804 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 0e938e1a-420c-4087-b8d1-93457eeb20e4 2026-02-12T19:13:16.804 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0e938e1a-420c-4087-b8d1-93457eeb20e4' 2026-02-12T19:13:16.804 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:13:16.812 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBMJo5poYItMBAAS/LCz73Lv9amIZqWdihJsQ== 2026-02-12T19:13:16.812 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBMJo5poYItMBAAS/LCz73Lv9amIZqWdihJsQ=="}' 2026-02-12T19:13:16.812 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0e938e1a-420c-4087-b8d1-93457eeb20e4 -i td/ceph-helpers/0/new.json 2026-02-12T19:13:16.924 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:16.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:13:16.932 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 --mkfs --key AQBMJo5poYItMBAAS/LCz73Lv9amIZqWdihJsQ== --osd-uuid 0e938e1a-420c-4087-b8d1-93457eeb20e4 2026-02-12T19:13:16.949 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:16.941+0000 7fe32e99aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:16.950 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:16.941+0000 7fe32e99aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:16.951 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:16.941+0000 7fe32e99aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:16.952 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:16.941+0000 7fe32e99aa40 -1 bdev(0x557f4e8c8400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:13:16.952 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:16.941+0000 7fe32e99aa40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:13:17.737 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:13:17.737 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:13:17.738 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:13:17.739 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:13:17.739 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:13:17.855 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:13:17.856 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:13:17.856 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_max_scrubs=3 2026-02-12T19:13:17.857 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:13:17.857 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:13:17.857 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:13:17.882 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:17.873+0000 7fc433abaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:17.884 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:17.873+0000 7fc433abaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:17.885 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:17.877+0000 7fc433abaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:17.967 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:13:17.967 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:13:17.968 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:13:17.968 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:13:17.968 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:13:17.968 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:17.968 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:13:17.968 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:17.969 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:17.969 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:18.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:18.450 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:18.441+0000 7fc433abaa40 -1 Falling back to public interface 2026-02-12T19:13:19.079 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:19.079 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:19.079 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:13:19.080 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:13:19.080 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:19.080 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:19.288 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:19.409 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:19.401+0000 7fc433abaa40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:13:20.290 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:20.290 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:20.290 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:13:20.290 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:13:20.290 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:20.290 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:20.507 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1757765603,v1:127.0.0.1:6801/1757765603] [v2:127.0.0.1:6802/1757765603,v1:127.0.0.1:6803/1757765603] exists,up 0e938e1a-420c-4087-b8d1-93457eeb20e4 2026-02-12T19:13:20.507 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:13:20.507 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:13:20.507 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:13:20.508 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: get_config osd 0 osd_max_scrubs 2026-02-12T19:13:20.508 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-12T19:13:20.508 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-12T19:13:20.508 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_max_scrubs 2026-02-12T19:13:20.508 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_max_scrubs 2026-02-12T19:13:20.508 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-12T19:13:20.509 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:13:20.509 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:13:20.509 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:20.509 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:20.509 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:20.509 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-osd.0.asok 2026-02-12T19:13:20.509 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:20.509 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-osd.0.asok config get osd_max_scrubs 2026-02-12T19:13:20.570 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1135: test_get_config: test 3 = 3 2026-02-12T19:13:20.570 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1136: test_get_config: teardown td/ceph-helpers 2026-02-12T19:13:20.570 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:20.570 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:20.570 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:20.570 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:20.570 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:20.570 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:20.570 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:20.570 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:20.678 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:20.678 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:20.679 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:20.680 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:20.681 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:20.681 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:20.681 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:20.682 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:20.682 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:20.683 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:20.683 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:20.684 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:20.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:20.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:20.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:20.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:20.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:20.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_is_making_recovery_progress td/ceph-helpers 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: test_get_is_making_recovery_progress: local dir=td/ceph-helpers 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: test_get_is_making_recovery_progress: setup td/ceph-helpers 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:20.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:20.693 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:20.693 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:20.694 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:20.694 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:20.695 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:20.695 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:20.695 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:20.696 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:20.696 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:20.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:20.697 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:20.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:20.699 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:20.699 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:20.700 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:20.700 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:20.700 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:20.700 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:20.701 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:20.701 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:20.701 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:20.703 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:20.703 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:20.703 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:20.703 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:20.704 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:20.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:20.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:20.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:20.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1338: test_get_is_making_recovery_progress: run_mon td/ceph-helpers a 2026-02-12T19:13:20.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:13:20.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:13:20.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:13:20.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:13:20.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:13:20.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-12T19:13:20.727 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:13:20.727 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:20.727 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:20.727 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:20.727 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:20.727 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:20.727 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:20.728 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:13:20.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:13:20.759 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:13:20.759 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:20.759 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:20.759 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:13:20.760 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:13:20.760 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:20.760 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:20.760 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:20.760 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:20.760 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:20.761 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:20.761 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:20.761 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:20.761 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:13:20.817 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:13:20.817 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:20.817 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:20.817 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:13:20.817 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:13:20.817 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:20.817 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:20.818 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:20.818 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:20.818 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:20.818 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:20.818 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:20.818 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:20.819 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:13:20.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: test_get_is_making_recovery_progress: run_mgr td/ceph-helpers x 2026-02-12T19:13:20.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:13:20.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:13:20.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:13:20.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:13:20.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:13:20.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:13:20.986 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:13:20.986 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:20.986 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:20.987 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:20.987 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:20.987 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:20.987 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:20.988 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:20.988 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:21.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: test_get_is_making_recovery_progress: get_is_making_recovery_progress 2026-02-12T19:13:21.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-12T19:13:21.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-12T19:13:21.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-12T19:13:21.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-12T19:13:21.008 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-12T19:13:21.008 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-12T19:13:21.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-12T19:13:21.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-12T19:13:21.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1341: test_get_is_making_recovery_progress: teardown td/ceph-helpers 2026-02-12T19:13:21.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:21.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:21.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:21.123 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:21.124 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:21.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:21.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:21.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:21.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:21.230 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:21.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:21.231 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:21.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:21.233 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:21.233 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:21.234 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:21.234 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:21.235 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:21.235 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:21.236 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:21.237 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:21.237 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:21.239 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:21.240 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:21.240 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:21.240 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:21.242 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:21.242 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:21.242 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:21.242 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_last_scrub_stamp td/ceph-helpers 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1524: test_get_last_scrub_stamp: local dir=td/ceph-helpers 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1526: test_get_last_scrub_stamp: setup td/ceph-helpers 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:21.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:21.244 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:21.244 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:21.245 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:21.245 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:21.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:21.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:21.247 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:21.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:21.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:21.248 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:21.248 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:21.249 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:21.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:21.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:21.251 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:21.251 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:21.251 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:21.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:21.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:21.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:21.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:21.254 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:21.254 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:21.254 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:21.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:21.255 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:21.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:21.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:21.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:21.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1527: test_get_last_scrub_stamp: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:21.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:13:21.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:13:21.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:13:21.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:13:21.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:13:21.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:21.278 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:13:21.278 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:21.278 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:21.278 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:21.279 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:21.279 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:21.279 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:21.279 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:21.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:13:21.308 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:13:21.308 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:21.308 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:21.308 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:13:21.309 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:13:21.309 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:21.309 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:21.309 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:21.309 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:21.309 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:21.310 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:21.310 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:21.310 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:21.310 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:13:21.364 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:13:21.364 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:21.364 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:21.364 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:13:21.364 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:13:21.365 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:21.365 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:21.365 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:21.366 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:21.366 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:21.366 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:21.366 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:21.366 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:21.366 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:13:21.419 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1528: test_get_last_scrub_stamp: run_mgr td/ceph-helpers x 2026-02-12T19:13:21.419 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:13:21.420 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:13:21.420 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:13:21.420 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:13:21.420 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:13:21.420 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:13:21.539 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:13:21.539 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:21.539 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:21.539 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:21.539 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:21.539 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:21.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:21.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:21.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:21.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1529: test_get_last_scrub_stamp: run_osd td/ceph-helpers 0 2026-02-12T19:13:21.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:13:21.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:13:21.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:13:21.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:13:21.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:13:21.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:13:21.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:13:21.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:13:21.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:13:21.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:13:21.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:13:21.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:13:21.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:13:21.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:13:21.562 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:13:21.562 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:13:21.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:13:21.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:13:21.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:13:21.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:13:21.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:13:21.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:13:21.564 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:13:21.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b4b44ea7-97b5-42a2-9db3-e496a77fa4a8 2026-02-12T19:13:21.565 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 b4b44ea7-97b5-42a2-9db3-e496a77fa4a8 2026-02-12T19:13:21.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b4b44ea7-97b5-42a2-9db3-e496a77fa4a8' 2026-02-12T19:13:21.565 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:13:21.574 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBRJo5pAFL4IRAArLlsU/ifbVVC3WPbKUIMjQ== 2026-02-12T19:13:21.574 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBRJo5pAFL4IRAArLlsU/ifbVVC3WPbKUIMjQ=="}' 2026-02-12T19:13:21.574 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b4b44ea7-97b5-42a2-9db3-e496a77fa4a8 -i td/ceph-helpers/0/new.json 2026-02-12T19:13:21.685 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:21.693 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:13:21.695 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBRJo5pAFL4IRAArLlsU/ifbVVC3WPbKUIMjQ== --osd-uuid b4b44ea7-97b5-42a2-9db3-e496a77fa4a8 2026-02-12T19:13:21.710 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:21.701+0000 7fd46c579a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:21.712 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:21.701+0000 7fd46c579a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:21.713 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:21.705+0000 7fd46c579a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:21.713 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:21.705+0000 7fd46c579a40 -1 bdev(0x559a8defa400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:13:21.714 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:21.705+0000 7fd46c579a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:13:22.451 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:13:22.451 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:13:22.452 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:13:22.452 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:13:22.452 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:13:22.579 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:13:22.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:13:22.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:13:22.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:13:22.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:13:22.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:13:22.603 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:22.593+0000 7fb7a0219a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:22.605 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:22.597+0000 7fb7a0219a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:22.607 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:22.597+0000 7fb7a0219a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:22.691 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:13:22.691 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:13:22.691 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:13:22.691 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:13:22.691 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:13:22.691 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:22.692 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:22.692 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:13:22.692 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:22.692 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:22.802 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:23.106 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:23.097+0000 7fb7a0219a40 -1 Falling back to public interface 2026-02-12T19:13:23.804 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:23.804 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:23.805 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:13:23.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:13:23.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:23.805 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:23.914 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:24.052 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:24.045+0000 7fb7a0219a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:13:24.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:24.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:24.916 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:13:24.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:13:24.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:24.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:25.136 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2822109859,v1:127.0.0.1:6801/2822109859] [v2:127.0.0.1:6802/2822109859,v1:127.0.0.1:6803/2822109859] exists,up b4b44ea7-97b5-42a2-9db3-e496a77fa4a8 2026-02-12T19:13:25.136 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:13:25.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:13:25.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:13:25.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1530: test_get_last_scrub_stamp: create_rbd_pool 2026-02-12T19:13:25.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:13:25.343 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:13:25.353 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:13:25.353 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:13:25.632 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:13:25.643 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:13:26.644 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:13:26.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1531: test_get_last_scrub_stamp: wait_for_clean 2026-02-12T19:13:26.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:13:26.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:13:26.921 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:13:26.921 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:13:26.921 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:13:26.921 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:13:26.922 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:13:26.922 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:13:26.922 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:13:27.024 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:13:27.024 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:13:27.024 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:13:27.024 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:13:27.024 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:13:27.024 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:13:27.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:13:27.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:13:27.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:13:27.247 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:13:27.325 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-12T19:13:27.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-12T19:13:27.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-12T19:13:27.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:13:27.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-12T19:13:27.326 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:13:27.327 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:13:27.328 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-12T19:13:27.328 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:13:27.329 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-12T19:13:27.330 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836483 2026-02-12T19:13:27.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-12T19:13:27.330 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:13:27.548 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-12T19:13:27.549 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:13:28.549 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:13:28.550 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:13:28.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-12T19:13:28.761 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:13:28.762 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:13:28.762 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:13:29.039 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:13:29.039 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:13:29.039 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:13:29.039 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:13:29.039 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:13:29.039 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:13:29.040 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:13:29.040 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:13:29.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:13:29.248 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:13:29.248 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:13:29.248 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:13:29.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:13:29.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:13:29.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:13:29.525 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: get_last_scrub_stamp 1.0 2026-02-12T19:13:29.525 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-12T19:13:29.525 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-12T19:13:29.525 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-12T19:13:29.525 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-12T19:13:29.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1532: test_get_last_scrub_stamp: stamp=2026-02-12T19:13:25.574283+0000 2026-02-12T19:13:29.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1533: test_get_last_scrub_stamp: test -n 2026-02-12T19:13:25.574283+0000 2026-02-12T19:13:29.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1534: test_get_last_scrub_stamp: teardown td/ceph-helpers 2026-02-12T19:13:29.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:29.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:29.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:29.751 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:29.751 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:29.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:29.752 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:29.753 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:29.858 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:29.858 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:29.859 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:29.859 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:29.860 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:29.860 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:29.860 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:29.861 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:29.861 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:29.861 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:29.862 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:29.862 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:29.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:29.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:29.868 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:29.868 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:29.868 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:29.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:29.870 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:29.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:29.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:29.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_not_primary td/ceph-helpers 2026-02-12T19:13:29.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1229: test_get_not_primary: local dir=td/ceph-helpers 2026-02-12T19:13:29.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1231: test_get_not_primary: setup td/ceph-helpers 2026-02-12T19:13:29.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:29.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:29.871 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:29.872 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:29.872 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:29.872 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:29.872 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:29.872 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:29.872 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:29.872 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:29.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:29.873 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:29.874 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:29.874 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:29.875 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:29.875 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:29.875 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:29.876 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:29.876 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:29.876 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:29.877 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:29.877 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:29.878 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:29.878 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:29.880 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:29.880 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:29.880 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:29.880 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:29.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:29.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:29.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:29.882 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:29.882 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:29.883 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:29.883 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:29.884 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:29.885 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:29.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:29.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:29.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1232: test_get_not_primary: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-12T19:13:29.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:13:29.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:13:29.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:13:29.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:13:29.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:13:29.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-12T19:13:29.907 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:13:29.908 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:29.908 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:29.908 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:29.908 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:29.908 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:29.908 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:29.909 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-12T19:13:29.936 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:13:29.937 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:13:29.937 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:29.937 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:29.937 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:13:29.938 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:13:29.938 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:29.939 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:29.939 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:29.939 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:29.939 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:29.939 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:29.939 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:29.939 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:29.939 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:13:29.994 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:13:29.994 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:29.994 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:29.995 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:13:29.996 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:13:29.996 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:29.996 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:29.996 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:29.996 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:29.996 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:29.996 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:29.996 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:29.996 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:29.997 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:13:30.055 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1233: test_get_not_primary: run_mgr td/ceph-helpers x 2026-02-12T19:13:30.055 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:13:30.055 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:13:30.055 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:13:30.055 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:13:30.055 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:13:30.055 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:13:30.172 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:13:30.173 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:30.173 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:30.173 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:30.173 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:30.173 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:30.173 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:30.174 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:30.174 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:30.194 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1234: test_get_not_primary: run_osd td/ceph-helpers 0 2026-02-12T19:13:30.194 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:13:30.194 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:13:30.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:13:30.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:13:30.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:13:30.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:13:30.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:13:30.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:13:30.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:13:30.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:13:30.197 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:13:30.197 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:13:30.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b80c2f4c-19e7-46e0-ac09-164884913cd5 2026-02-12T19:13:30.198 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 b80c2f4c-19e7-46e0-ac09-164884913cd5 2026-02-12T19:13:30.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b80c2f4c-19e7-46e0-ac09-164884913cd5' 2026-02-12T19:13:30.198 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:13:30.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBaJo5pXwALDBAAH7RwjuRSd0Vti7GvfOO4AQ== 2026-02-12T19:13:30.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBaJo5pXwALDBAAH7RwjuRSd0Vti7GvfOO4AQ=="}' 2026-02-12T19:13:30.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b80c2f4c-19e7-46e0-ac09-164884913cd5 -i td/ceph-helpers/0/new.json 2026-02-12T19:13:30.316 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:30.324 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:13:30.325 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBaJo5pXwALDBAAH7RwjuRSd0Vti7GvfOO4AQ== --osd-uuid b80c2f4c-19e7-46e0-ac09-164884913cd5 2026-02-12T19:13:30.341 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:30.333+0000 7f8f50a9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:30.343 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:30.333+0000 7f8f50a9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:30.344 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:30.333+0000 7f8f50a9ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:30.345 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:30.337+0000 7f8f50a9ea40 -1 bdev(0x555987118400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:13:30.345 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:30.337+0000 7f8f50a9ea40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:13:31.164 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:13:31.165 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:13:31.165 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:13:31.166 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:13:31.166 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:13:31.277 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:13:31.278 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:13:31.278 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:13:31.278 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:13:31.278 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:13:31.278 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:13:31.293 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:31.289+0000 7f4adbee4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:31.294 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:31.289+0000 7f4adbee4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:31.295 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:31.289+0000 7f4adbee4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:31.391 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:13:31.391 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:13:31.391 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:13:31.392 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:13:31.392 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:13:31.392 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:31.392 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:31.393 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:13:31.393 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:31.393 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:31.498 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:31.702 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:31.697+0000 7f4adbee4a40 -1 Falling back to public interface 2026-02-12T19:13:32.443 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:32.437+0000 7f4adbee4a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:13:32.500 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:32.500 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:32.500 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:13:32.501 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:13:32.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:32.501 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:32.612 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:33.615 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:33.615 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:33.615 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:13:33.616 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:13:33.616 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:33.616 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:33.833 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/249489919,v1:127.0.0.1:6801/249489919] [v2:127.0.0.1:6802/249489919,v1:127.0.0.1:6803/249489919] exists,up b80c2f4c-19e7-46e0-ac09-164884913cd5 2026-02-12T19:13:33.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:13:33.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:13:33.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:13:33.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1235: test_get_not_primary: run_osd td/ceph-helpers 1 2026-02-12T19:13:33.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:13:33.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:13:33.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T19:13:33.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:13:33.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-12T19:13:33.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:13:33.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:13:33.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:13:33.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:13:33.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-12T19:13:33.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-12T19:13:33.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:13:33.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:13:33.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:13:33.839 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:13:33.839 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:33.839 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:33.839 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:33.839 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:33.839 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:33.839 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:33.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:33.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:13:33.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:13:33.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:13:33.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:13:33.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:13:33.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:13:33.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:13:33.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:13:33.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:13:33.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:13:33.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:13:33.841 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-12T19:13:33.841 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:13:33.841 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b6ac7acc-94f8-442e-aba0-8b7a210e665e 2026-02-12T19:13:33.841 INFO:tasks.workunit.client.0.trial010.stdout:add osd1 b6ac7acc-94f8-442e-aba0-8b7a210e665e 2026-02-12T19:13:33.842 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b6ac7acc-94f8-442e-aba0-8b7a210e665e' 2026-02-12T19:13:33.842 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:13:33.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBdJo5pG8hlMhAALanvZViOsTgT9eRqiEDmSw== 2026-02-12T19:13:33.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBdJo5pG8hlMhAALanvZViOsTgT9eRqiEDmSw=="}' 2026-02-12T19:13:33.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b6ac7acc-94f8-442e-aba0-8b7a210e665e -i td/ceph-helpers/1/new.json 2026-02-12T19:13:34.074 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:13:34.083 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-12T19:13:34.084 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBdJo5pG8hlMhAALanvZViOsTgT9eRqiEDmSw== --osd-uuid b6ac7acc-94f8-442e-aba0-8b7a210e665e 2026-02-12T19:13:34.101 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:34.097+0000 7f1cc8ceaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:34.103 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:34.097+0000 7f1cc8ceaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:34.104 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:34.097+0000 7f1cc8ceaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:34.104 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:34.097+0000 7f1cc8ceaa40 -1 bdev(0x55c38d2e2400 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-12T19:13:34.105 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:34.101+0000 7f1cc8ceaa40 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-12T19:13:34.779 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-12T19:13:34.779 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:13:34.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T19:13:34.781 INFO:tasks.workunit.client.0.trial010.stdout:adding osd1 key to auth repository 2026-02-12T19:13:34.781 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:13:35.073 INFO:tasks.workunit.client.0.trial010.stdout:start osd.1 2026-02-12T19:13:35.073 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T19:13:35.074 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:13:35.074 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:13:35.074 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:13:35.074 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:13:35.090 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:35.085+0000 7f18b836ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:35.091 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:35.085+0000 7f18b836ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:35.092 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:35.089+0000 7f18b836ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:35.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T19:13:35.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:13:35.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T19:13:35.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:13:35.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:13:35.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:35.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:13:35.287 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:35.287 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:35.287 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:13:35.478 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:35.473+0000 7f18b836ca40 -1 Falling back to public interface 2026-02-12T19:13:35.507 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:36.073 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:36.069+0000 7f18b836ca40 -1 osd.1 0 log_to_monitors true 2026-02-12T19:13:36.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:36.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:36.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:13:36.509 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:13:36.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:36.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:13:36.723 INFO:tasks.workunit.client.0.trial010.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1916629648,v1:127.0.0.1:6811/1916629648] [v2:127.0.0.1:6812/1916629648,v1:127.0.0.1:6813/1916629648] exists,up b6ac7acc-94f8-442e-aba0-8b7a210e665e 2026-02-12T19:13:36.723 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:13:36.724 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:13:36.724 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:13:36.724 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1236: test_get_not_primary: create_rbd_pool 2026-02-12T19:13:36.724 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:13:36.933 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:13:36.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:13:36.942 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:13:37.256 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:13:37.265 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:13:38.266 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:13:38.548 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1237: test_get_not_primary: wait_for_clean 2026-02-12T19:13:38.548 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:13:38.548 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:13:38.549 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:13:38.549 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:13:38.550 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:13:38.550 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:13:38.550 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:13:38.550 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:13:38.550 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:13:38.654 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:13:38.654 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:13:38.654 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:13:38.654 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:13:38.654 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:13:38.654 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:13:38.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T19:13:38.873 INFO:tasks.workunit.client.0.trial010.stderr:1' 2026-02-12T19:13:38.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:13:38.873 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:13:38.874 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:13:38.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-12T19:13:38.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-12T19:13:38.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-12T19:13:38.954 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:13:38.954 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T19:13:39.034 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-12T19:13:39.034 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-12T19:13:39.035 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672964' 2026-02-12T19:13:39.035 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:13:39.035 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-12T19:13:39.035 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:13:39.036 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:13:39.037 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-12T19:13:39.037 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:13:39.038 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-12T19:13:39.038 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-12T19:13:39.039 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836484 2026-02-12T19:13:39.039 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:13:39.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836484 2026-02-12T19:13:39.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:13:40.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:13:40.254 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:13:40.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836484 2026-02-12T19:13:40.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:13:41.466 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T19:13:41.467 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:13:41.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-12T19:13:41.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:13:41.689 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672964 2026-02-12T19:13:41.689 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:13:41.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T19:13:41.690 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-12T19:13:41.690 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:13:41.691 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672964 2026-02-12T19:13:41.692 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.1 seq 42949672964 2026-02-12T19:13:41.692 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-12T19:13:41.692 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T19:13:41.915 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-12T19:13:41.915 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:13:41.916 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:13:41.916 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:13:42.194 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:13:42.194 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:13:42.194 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:13:42.194 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:13:42.194 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:13:42.194 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:13:42.195 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:13:42.195 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:13:42.402 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:13:42.402 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:13:42.403 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:13:42.403 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:13:42.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:13:42.687 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:13:42.687 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:13:42.687 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: get_primary rbd GROUP 2026-02-12T19:13:42.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-12T19:13:42.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-12T19:13:42.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-12T19:13:42.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-12T19:13:42.905 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1238: test_get_not_primary: local primary=1 2026-02-12T19:13:42.906 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: get_not_primary rbd GROUP 2026-02-12T19:13:42.906 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1220: get_not_primary: local poolname=rbd 2026-02-12T19:13:42.906 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1221: get_not_primary: local objectname=GROUP 2026-02-12T19:13:42.906 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: get_primary rbd GROUP 2026-02-12T19:13:42.906 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-12T19:13:42.906 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-12T19:13:42.906 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-12T19:13:42.907 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-12T19:13:43.129 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1223: get_not_primary: local primary=1 2026-02-12T19:13:43.129 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1224: get_not_primary: ceph --format json osd map rbd GROUP 2026-02-12T19:13:43.129 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1225: get_not_primary: jq '.acting | map(select (. != 1)) | .[0]' 2026-02-12T19:13:43.342 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1239: test_get_not_primary: local not_primary=0 2026-02-12T19:13:43.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1240: test_get_not_primary: test 0 '!=' 1 2026-02-12T19:13:43.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1241: test_get_not_primary: test 0 = 0 -o 0 = 1 2026-02-12T19:13:43.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1242: test_get_not_primary: teardown td/ceph-helpers 2026-02-12T19:13:43.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:43.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:43.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:43.343 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:43.343 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:43.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:43.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:43.344 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:43.451 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:43.451 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:43.451 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:43.452 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:43.453 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:43.453 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:43.453 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:43.454 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:43.454 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:43.455 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:43.455 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:43.456 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:43.457 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:43.457 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:43.462 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:43.462 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:43.463 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:43.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:43.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:43.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:43.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:43.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_active_clean td/ceph-helpers 2026-02-12T19:13:43.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1363: test_get_num_active_clean: local dir=td/ceph-helpers 2026-02-12T19:13:43.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: test_get_num_active_clean: setup td/ceph-helpers 2026-02-12T19:13:43.465 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:43.466 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:43.466 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:43.466 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:43.466 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:43.466 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:43.466 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:43.466 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:43.466 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:43.466 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:43.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:43.467 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:43.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:43.468 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:43.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:43.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:43.470 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:43.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:43.471 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:43.471 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:43.471 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:43.472 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:43.473 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:43.474 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:43.475 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:43.475 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:43.475 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:43.475 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:43.476 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:43.476 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:43.476 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:43.478 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:43.478 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:43.478 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:43.478 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:43.481 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:43.481 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:43.481 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:43.481 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:43.481 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: test_get_num_active_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:43.481 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:13:43.481 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:13:43.481 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:13:43.481 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:13:43.481 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:13:43.482 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:43.502 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:13:43.503 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:43.503 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:43.503 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:43.503 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:43.503 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:43.503 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:43.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:43.532 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:13:43.533 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:13:43.533 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:43.533 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:43.534 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:13:43.535 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:13:43.535 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:43.535 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:43.535 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:43.535 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:43.535 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:43.535 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:43.535 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:43.535 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:43.535 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:13:43.591 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:13:43.591 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:43.591 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:43.591 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:13:43.592 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:13:43.593 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:43.593 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:43.593 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:43.593 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:43.593 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:43.593 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:43.593 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:43.593 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:43.593 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:13:43.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: test_get_num_active_clean: run_mgr td/ceph-helpers x 2026-02-12T19:13:43.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:13:43.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:13:43.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:13:43.649 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:13:43.649 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:13:43.649 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:13:43.761 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:13:43.761 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:43.761 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:43.762 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:43.762 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:43.762 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:43.762 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:43.762 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:43.763 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:43.784 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: test_get_num_active_clean: run_osd td/ceph-helpers 0 2026-02-12T19:13:43.784 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:13:43.784 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:13:43.784 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:13:43.784 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:13:43.784 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:13:43.784 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:13:43.784 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:43.785 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:13:43.786 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:13:43.787 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:13:43.787 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=29190910-d9c8-492b-ad25-53893aa3ddc8 2026-02-12T19:13:43.787 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 29190910-d9c8-492b-ad25-53893aa3ddc8' 2026-02-12T19:13:43.787 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 29190910-d9c8-492b-ad25-53893aa3ddc8 2026-02-12T19:13:43.788 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:13:43.795 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBnJo5p9EIpLxAAWcDApoFYMPf4lnWpIp6HYw== 2026-02-12T19:13:43.795 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBnJo5p9EIpLxAAWcDApoFYMPf4lnWpIp6HYw=="}' 2026-02-12T19:13:43.796 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 29190910-d9c8-492b-ad25-53893aa3ddc8 -i td/ceph-helpers/0/new.json 2026-02-12T19:13:43.907 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:43.915 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:13:43.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBnJo5p9EIpLxAAWcDApoFYMPf4lnWpIp6HYw== --osd-uuid 29190910-d9c8-492b-ad25-53893aa3ddc8 2026-02-12T19:13:43.931 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:43.925+0000 7f5642d53a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:43.933 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:43.929+0000 7f5642d53a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:43.934 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:43.929+0000 7f5642d53a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:43.934 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:43.929+0000 7f5642d53a40 -1 bdev(0x55de47ca6400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:13:43.934 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:43.929+0000 7f5642d53a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:13:44.681 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:13:44.681 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:13:44.682 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:13:44.682 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:13:44.682 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:13:44.807 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:13:44.808 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:13:44.808 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:13:44.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:13:44.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:13:44.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:13:44.831 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:44.825+0000 7fe9f98eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:44.834 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:44.829+0000 7fe9f98eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:44.835 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:44.829+0000 7fe9f98eba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:44.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:13:44.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:13:44.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:13:44.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:13:44.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:13:44.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:44.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:13:44.919 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:44.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:44.919 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:45.030 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:45.290 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:45.285+0000 7fe9f98eba40 -1 Falling back to public interface 2026-02-12T19:13:46.032 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:46.032 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:46.032 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:13:46.033 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:13:46.033 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:46.033 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:46.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:46.150 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:46.145+0000 7fe9f98eba40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:13:47.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:47.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:47.138 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:13:47.139 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:13:47.139 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:47.139 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:47.354 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/847798435,v1:127.0.0.1:6801/847798435] [v2:127.0.0.1:6802/847798435,v1:127.0.0.1:6803/847798435] exists,up 29190910-d9c8-492b-ad25-53893aa3ddc8 2026-02-12T19:13:47.354 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:13:47.354 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:13:47.354 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:13:47.354 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1369: test_get_num_active_clean: create_rbd_pool 2026-02-12T19:13:47.354 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:13:47.558 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:13:47.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:13:47.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:13:47.840 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:13:47.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:13:48.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:13:49.145 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1370: test_get_num_active_clean: wait_for_clean 2026-02-12T19:13:49.145 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:13:49.145 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:13:49.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:13:49.146 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:13:49.146 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:13:49.147 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:13:49.147 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:13:49.147 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:13:49.147 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:13:49.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:13:49.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:13:49.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:13:49.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:13:49.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:13:49.247 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:13:49.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:13:49.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:13:49.463 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:13:49.464 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:13:49.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-12T19:13:49.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-12T19:13:49.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-12T19:13:49.544 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:13:49.545 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-12T19:13:49.545 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:13:49.545 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:13:49.546 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-12T19:13:49.546 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:13:49.547 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-12T19:13:49.548 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836483 2026-02-12T19:13:49.548 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-12T19:13:49.548 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:13:49.772 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-12T19:13:49.772 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:13:50.773 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:13:50.774 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:13:50.992 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-12T19:13:50.992 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:13:50.992 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:13:50.992 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:13:51.267 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:13:51.267 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:13:51.267 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:13:51.267 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:13:51.267 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:13:51.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:13:51.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:13:51.268 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:13:51.471 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:13:51.472 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:13:51.472 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:13:51.472 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:13:51.745 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:13:51.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:13:51.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:13:51.746 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: get_num_active_clean 2026-02-12T19:13:51.746 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:13:51.746 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:13:51.746 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:13:51.746 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:13:51.746 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:13:51.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1371: test_get_num_active_clean: local num_active_clean=4 2026-02-12T19:13:51.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1372: test_get_num_active_clean: test 4 = 4 2026-02-12T19:13:51.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1373: test_get_num_active_clean: teardown td/ceph-helpers 2026-02-12T19:13:51.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:51.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:51.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:51.953 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:51.953 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:51.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:51.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:51.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:52.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:52.062 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:52.062 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:52.063 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:52.064 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:52.064 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:52.065 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:52.066 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:52.066 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:52.066 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:52.066 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:52.068 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:52.069 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:52.069 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:52.074 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:52.074 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:52.074 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:52.074 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_active_or_peered td/ceph-helpers 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1393: test_get_num_active_or_peered: local dir=td/ceph-helpers 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1395: test_get_num_active_or_peered: setup td/ceph-helpers 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:13:52.076 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:13:52.077 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:13:52.077 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:13:52.077 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:13:52.079 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:13:52.079 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:13:52.080 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:13:52.080 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:13:52.081 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:13:52.081 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:13:52.081 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:13:52.082 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:52.083 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:13:52.083 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:13:52.083 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:13:52.084 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:13:52.085 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:13:52.085 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:13:52.087 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:13:52.087 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:52.087 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:52.087 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:13:52.088 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:13:52.088 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:13:52.088 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:13:52.089 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:13:52.089 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:52.089 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:52.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:13:52.091 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:13:52.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:13:52.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:13:52.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:13:52.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1396: test_get_num_active_or_peered: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:52.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:13:52.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:13:52.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:13:52.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:13:52.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:13:52.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:52.113 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:13:52.113 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:52.113 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:52.113 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:52.113 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:52.113 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:52.113 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:52.114 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:13:52.144 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:13:52.146 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:13:52.146 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:52.147 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:52.147 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:13:52.147 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:13:52.147 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:52.147 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:52.147 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:52.147 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:52.147 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:52.148 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:52.148 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:52.149 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:52.149 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:13:52.203 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:13:52.203 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:13:52.203 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:13:52.203 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:13:52.203 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:13:52.203 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:13:52.203 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:13:52.204 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:13:52.204 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:13:52.204 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:52.204 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:52.204 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:13:52.205 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:13:52.205 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:13:52.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1397: test_get_num_active_or_peered: run_mgr td/ceph-helpers x 2026-02-12T19:13:52.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:13:52.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:13:52.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:13:52.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:13:52.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:13:52.258 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:13:52.371 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:13:52.371 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:52.371 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:52.372 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:52.372 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:52.372 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:52.372 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:52.372 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:52.373 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:13:52.395 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1398: test_get_num_active_or_peered: run_osd td/ceph-helpers 0 2026-02-12T19:13:52.395 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:13:52.395 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:13:52.395 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:13:52.395 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:13:52.395 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:13:52.395 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:13:52.396 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:13:52.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:13:52.398 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:13:52.398 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:13:52.398 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:13:52.398 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:13:52.398 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=cd89bfea-9be9-4860-9c3f-2ffac33f84f1 2026-02-12T19:13:52.399 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 cd89bfea-9be9-4860-9c3f-2ffac33f84f1' 2026-02-12T19:13:52.399 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 cd89bfea-9be9-4860-9c3f-2ffac33f84f1 2026-02-12T19:13:52.399 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:13:52.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBwJo5p/k0pGBAA6uAR/F8Co5swhx8q5TeImQ== 2026-02-12T19:13:52.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBwJo5p/k0pGBAA6uAR/F8Co5swhx8q5TeImQ=="}' 2026-02-12T19:13:52.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new cd89bfea-9be9-4860-9c3f-2ffac33f84f1 -i td/ceph-helpers/0/new.json 2026-02-12T19:13:52.522 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:52.531 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:13:52.532 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBwJo5p/k0pGBAA6uAR/F8Co5swhx8q5TeImQ== --osd-uuid cd89bfea-9be9-4860-9c3f-2ffac33f84f1 2026-02-12T19:13:52.548 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:52.540+0000 7f6e4efd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:52.549 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:52.544+0000 7f6e4efd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:52.550 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:52.544+0000 7f6e4efd1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:52.551 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:52.544+0000 7f6e4efd1a40 -1 bdev(0x55f3358aa400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:13:52.551 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:52.544+0000 7f6e4efd1a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:13:53.262 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:13:53.263 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:13:53.263 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:13:53.264 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:13:53.264 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:13:53.380 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:13:53.381 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:13:53.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:13:53.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:13:53.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:13:53.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:13:53.397 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:53.392+0000 7f486fb41a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:53.398 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:53.392+0000 7f486fb41a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:53.400 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:53.392+0000 7f486fb41a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:13:53.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:13:53.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:13:53.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:13:53.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:13:53.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:13:53.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:53.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:13:53.496 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:13:53.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:53.497 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:53.605 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:53.882 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:53.876+0000 7f486fb41a40 -1 Falling back to public interface 2026-02-12T19:13:54.606 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:54.607 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:13:54.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:54.607 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:13:54.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:54.608 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:54.658 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:13:54.652+0000 7f486fb41a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:13:54.713 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:13:55.715 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:13:55.715 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:13:55.715 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:13:55.716 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:13:55.716 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:13:55.716 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:13:55.928 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/656556782,v1:127.0.0.1:6801/656556782] [v2:127.0.0.1:6802/656556782,v1:127.0.0.1:6803/656556782] exists,up cd89bfea-9be9-4860-9c3f-2ffac33f84f1 2026-02-12T19:13:55.929 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:13:55.929 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:13:55.929 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:13:55.929 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1399: test_get_num_active_or_peered: create_rbd_pool 2026-02-12T19:13:55.929 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:13:56.130 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:13:56.139 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:13:56.140 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:13:56.415 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:13:56.424 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:13:57.425 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:13:57.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1400: test_get_num_active_or_peered: wait_for_clean 2026-02-12T19:13:57.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:13:57.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:13:57.705 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:13:57.705 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:13:57.706 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:13:57.707 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:13:57.707 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:13:57.707 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:13:57.707 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:13:57.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:13:57.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:13:57.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:13:57.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:13:57.807 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:13:57.807 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:13:58.027 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:13:58.028 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:13:58.028 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:13:58.028 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:13:58.109 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-12T19:13:58.109 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-12T19:13:58.109 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-12T19:13:58.109 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:13:58.110 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-12T19:13:58.110 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:13:58.110 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:13:58.111 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-12T19:13:58.111 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:13:58.112 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-12T19:13:58.112 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-12T19:13:58.113 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836483 2026-02-12T19:13:58.113 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:13:58.325 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-12T19:13:58.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:13:59.327 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:13:59.327 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:13:59.547 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-12T19:13:59.548 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:13:59.548 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:13:59.548 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:13:59.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:13:59.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:13:59.839 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:13:59.840 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:13:59.840 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:13:59.840 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:13:59.840 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:13:59.840 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:14:00.054 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:14:00.055 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:14:00.055 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:00.055 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:00.344 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:14:00.344 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:14:00.344 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:14:00.345 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: get_num_active_or_peered 2026-02-12T19:14:00.346 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1385: get_num_active_or_peered: local expression 2026-02-12T19:14:00.346 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1386: get_num_active_or_peered: expression+='select(contains("active") or contains("peered")) | ' 2026-02-12T19:14:00.346 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1387: get_num_active_or_peered: expression+='select(contains("stale") | not)' 2026-02-12T19:14:00.346 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1388: get_num_active_or_peered: ceph --format json pg dump pgs 2026-02-12T19:14:00.346 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1389: get_num_active_or_peered: jq '.pg_stats | [.[] | .state | select(contains("active") or contains("peered")) | select(contains("stale") | not)] | length' 2026-02-12T19:14:00.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1401: test_get_num_active_or_peered: local num_peered=4 2026-02-12T19:14:00.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1402: test_get_num_active_or_peered: test 4 = 4 2026-02-12T19:14:00.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1403: test_get_num_active_or_peered: teardown td/ceph-helpers 2026-02-12T19:14:00.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:00.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:00.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:00.563 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:00.563 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:00.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:00.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:00.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:00.670 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:00.671 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:00.671 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:00.672 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:00.674 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:00.674 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:00.674 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:00.675 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:00.675 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:00.675 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:00.675 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:00.676 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:00.677 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:00.677 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:00.683 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:00.683 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:00.683 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:00.683 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:00.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:00.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:00.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:14:00.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_num_pgs td/ceph-helpers 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1420: test_get_num_pgs: local dir=td/ceph-helpers 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1422: test_get_num_pgs: setup td/ceph-helpers 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:00.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:00.687 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:00.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:00.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:00.689 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:00.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:00.690 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:00.690 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:00.691 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:00.691 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:00.692 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:00.692 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:00.693 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:00.694 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:00.694 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:00.695 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:00.696 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:00.696 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:00.696 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:00.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:00.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:00.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:14:00.698 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:14:00.698 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:00.698 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:00.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:14:00.700 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:14:00.701 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:14:00.701 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:14:00.701 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:14:00.701 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1423: test_get_num_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:00.701 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:14:00.701 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:14:00.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:14:00.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:14:00.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:14:00.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:00.722 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:14:00.722 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:00.722 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:00.722 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:00.722 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:00.722 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:00.722 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:00.723 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:00.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:14:00.752 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:14:00.752 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:00.752 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:00.753 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:14:00.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:14:00.754 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:00.754 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:00.754 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:00.754 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:00.754 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:00.754 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:00.754 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:00.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:00.755 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:14:00.809 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:14:00.809 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:00.809 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:00.809 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:14:00.810 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:14:00.810 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:00.811 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:00.811 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:00.811 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:00.811 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:00.811 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:00.811 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:00.811 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:00.811 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:14:00.865 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1424: test_get_num_pgs: run_mgr td/ceph-helpers x 2026-02-12T19:14:00.865 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:14:00.865 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:14:00.865 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:14:00.866 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:14:00.866 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:14:00.866 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:14:00.977 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:14:00.977 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:00.977 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:00.978 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:00.978 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:00.978 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:00.978 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:00.978 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:00.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:01.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: test_get_num_pgs: run_osd td/ceph-helpers 0 2026-02-12T19:14:01.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:14:01.000 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:14:01.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:14:01.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:14:01.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:14:01.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:14:01.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:14:01.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:14:01.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:14:01.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:14:01.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:14:01.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:14:01.001 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:14:01.002 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:14:01.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:14:01.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:14:01.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:14:01.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:14:01.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:14:01.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:14:01.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:14:01.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:14:01.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:14:01.003 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:14:01.004 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=120ce50a-4844-4021-8d43-09a04563f8b4 2026-02-12T19:14:01.004 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 120ce50a-4844-4021-8d43-09a04563f8b4' 2026-02-12T19:14:01.004 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 120ce50a-4844-4021-8d43-09a04563f8b4 2026-02-12T19:14:01.005 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:14:01.013 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB5Jo5ptt+KABAAPj9SamGFhPYVMfMCnL+QNA== 2026-02-12T19:14:01.013 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB5Jo5ptt+KABAAPj9SamGFhPYVMfMCnL+QNA=="}' 2026-02-12T19:14:01.013 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 120ce50a-4844-4021-8d43-09a04563f8b4 -i td/ceph-helpers/0/new.json 2026-02-12T19:14:01.128 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:01.136 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:14:01.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB5Jo5ptt+KABAAPj9SamGFhPYVMfMCnL+QNA== --osd-uuid 120ce50a-4844-4021-8d43-09a04563f8b4 2026-02-12T19:14:01.153 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:01.148+0000 7f68a1a4ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:01.155 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:01.148+0000 7f68a1a4ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:01.156 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:01.148+0000 7f68a1a4ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:01.156 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:01.148+0000 7f68a1a4ba40 -1 bdev(0x55f9264c0400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:14:01.156 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:01.148+0000 7f68a1a4ba40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:14:01.815 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:14:01.815 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:14:01.816 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:14:01.817 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:14:01.817 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:14:01.933 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:14:01.933 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:14:01.934 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:14:01.934 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:14:01.934 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:14:01.934 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:14:01.950 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:01.944+0000 7fc867655a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:01.952 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:01.944+0000 7fc867655a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:01.953 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:01.948+0000 7fc867655a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:02.046 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:14:02.047 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:14:02.047 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:14:02.047 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:14:02.047 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:14:02.047 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:02.047 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:02.048 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:14:02.048 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:02.048 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:02.158 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:02.338 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:02.332+0000 7fc867655a40 -1 Falling back to public interface 2026-02-12T19:14:03.034 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:03.028+0000 7fc867655a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:14:03.159 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:03.159 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:03.160 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:14:03.160 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:14:03.163 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:03.163 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:03.284 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:04.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:04.285 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:04.286 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:14:04.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:14:04.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:04.287 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:04.495 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1175499941,v1:127.0.0.1:6801/1175499941] [v2:127.0.0.1:6802/1175499941,v1:127.0.0.1:6803/1175499941] exists,up 120ce50a-4844-4021-8d43-09a04563f8b4 2026-02-12T19:14:04.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:14:04.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:14:04.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:14:04.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1426: test_get_num_pgs: create_rbd_pool 2026-02-12T19:14:04.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:14:04.703 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:14:04.715 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:14:04.715 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:14:04.939 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:14:04.950 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:14:05.952 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:14:06.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1427: test_get_num_pgs: wait_for_clean 2026-02-12T19:14:06.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:14:06.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:14:06.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:14:06.232 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:14:06.233 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:06.233 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:06.233 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:06.233 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:06.233 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:06.334 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:14:06.334 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:14:06.334 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:14:06.335 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:14:06.335 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:14:06.335 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:14:06.545 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:14:06.545 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:14:06.545 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:14:06.546 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:14:06.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-12T19:14:06.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-12T19:14:06.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-12T19:14:06.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:14:06.627 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-12T19:14:06.627 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:14:06.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:14:06.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-12T19:14:06.629 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:14:06.630 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-12T19:14:06.630 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836483 2026-02-12T19:14:06.630 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-12T19:14:06.630 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:14:06.855 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-12T19:14:06.855 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:14:07.856 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:14:07.856 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:14:08.067 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-12T19:14:08.068 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:14:08.068 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:08.068 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:08.344 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:14:08.344 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:14:08.345 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:14:08.345 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:14:08.345 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:14:08.345 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:14:08.345 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:14:08.345 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:14:08.553 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:14:08.554 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:14:08.554 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:08.554 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:08.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:14:08.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:14:08.835 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:14:08.835 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: get_num_pgs 2026-02-12T19:14:08.836 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:08.836 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:09.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1428: test_get_num_pgs: local num_pgs=4 2026-02-12T19:14:09.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1429: test_get_num_pgs: test 4 -gt 0 2026-02-12T19:14:09.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1430: test_get_num_pgs: teardown td/ceph-helpers 2026-02-12T19:14:09.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:09.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:09.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:09.124 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:09.124 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:09.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:09.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:09.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:09.231 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:09.231 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:09.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:09.233 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:09.234 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:09.234 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:09.234 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:09.235 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:09.235 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:09.236 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:09.236 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:09.237 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:09.238 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:09.238 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:09.243 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:09.243 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:09.243 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:09.243 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:09.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:09.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:09.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:14:09.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_op_scheduler td/ceph-helpers 2026-02-12T19:14:09.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2315: test_get_op_scheduler: local dir=td/ceph-helpers 2026-02-12T19:14:09.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2317: test_get_op_scheduler: setup td/ceph-helpers 2026-02-12T19:14:09.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:14:09.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:14:09.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:09.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:09.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:09.247 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:09.247 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:09.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:09.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:09.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:09.248 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:09.248 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:09.249 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:09.249 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:09.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:09.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:09.251 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:09.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:09.252 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:09.253 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:09.253 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:09.254 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:09.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:09.255 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:09.256 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:09.256 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:09.256 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:09.256 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:09.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:09.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:09.257 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:14:09.258 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:14:09.258 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:09.258 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:09.259 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:14:09.260 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:14:09.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:14:09.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:14:09.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:14:09.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2319: test_get_op_scheduler: run_mon td/ceph-helpers a 2026-02-12T19:14:09.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:14:09.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:14:09.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:14:09.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:14:09.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:14:09.261 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers 2026-02-12T19:14:09.282 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:14:09.282 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:09.282 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:09.282 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:09.282 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:09.282 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:09.283 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:09.283 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-12T19:14:09.311 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:14:09.312 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:14:09.312 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:09.312 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:09.313 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:14:09.314 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:14:09.314 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:09.314 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:09.314 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:09.314 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:09.314 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:09.314 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:09.314 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:09.315 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:09.315 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:14:09.369 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:14:09.370 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:09.370 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:09.370 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:14:09.370 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:14:09.371 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:09.371 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:09.371 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:09.371 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:09.371 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:09.371 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:09.372 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:09.372 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:09.372 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:14:09.426 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2320: test_get_op_scheduler: run_mgr td/ceph-helpers x 2026-02-12T19:14:09.426 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:14:09.426 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:14:09.426 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:14:09.426 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:14:09.426 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:14:09.426 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:14:09.536 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:14:09.537 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:09.537 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:09.537 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:09.537 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:09.538 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:09.538 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:09.538 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:09.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:09.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2322: test_get_op_scheduler: run_osd td/ceph-helpers 0 --osd_op_queue=wpq 2026-02-12T19:14:09.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:14:09.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:14:09.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:14:09.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:14:09.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:14:09.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:14:09.560 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:09.561 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:14:09.562 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:14:09.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=wpq 2026-02-12T19:14:09.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:14:09.563 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:14:09.563 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=05699e62-942f-4c78-b294-f0df41dceb2e 2026-02-12T19:14:09.564 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 05699e62-942f-4c78-b294-f0df41dceb2e 2026-02-12T19:14:09.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 05699e62-942f-4c78-b294-f0df41dceb2e' 2026-02-12T19:14:09.564 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:14:09.573 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCBJo5prQziIRAApL5OzUW3Yvt+dJn2OEqomw== 2026-02-12T19:14:09.573 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCBJo5prQziIRAApL5OzUW3Yvt+dJn2OEqomw=="}' 2026-02-12T19:14:09.573 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 05699e62-942f-4c78-b294-f0df41dceb2e -i td/ceph-helpers/0/new.json 2026-02-12T19:14:09.686 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:09.694 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:14:09.695 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq --mkfs --key AQCBJo5prQziIRAApL5OzUW3Yvt+dJn2OEqomw== --osd-uuid 05699e62-942f-4c78-b294-f0df41dceb2e 2026-02-12T19:14:09.712 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:09.704+0000 7f288b727a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:09.714 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:09.708+0000 7f288b727a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:09.715 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:09.708+0000 7f288b727a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:09.716 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:09.708+0000 7f288b727a40 -1 bdev(0x55d9803fa400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:14:09.716 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:09.708+0000 7f288b727a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:14:10.490 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:14:10.491 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:14:10.491 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:14:10.492 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:14:10.492 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:14:10.609 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:14:10.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:14:10.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=wpq 2026-02-12T19:14:10.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:14:10.611 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:14:10.611 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:14:10.635 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:10.628+0000 7fe5d1cdda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:10.637 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:10.632+0000 7fe5d1cdda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:10.638 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:10.632+0000 7fe5d1cdda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:10.719 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:14:10.719 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:14:10.719 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:14:10.720 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:14:10.720 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:14:10.720 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:10.720 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:14:10.720 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:10.721 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:10.721 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:10.824 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:11.090 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:11.084+0000 7fe5d1cdda40 -1 Falling back to public interface 2026-02-12T19:14:11.825 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:11.826 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:11.826 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:14:11.827 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:14:11.827 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:11.827 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:11.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:12.126 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:12.120+0000 7fe5d1cdda40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:14:12.933 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:12.933 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:12.934 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:14:12.934 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:14:12.934 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:12.934 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:13.150 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/501244270,v1:127.0.0.1:6801/501244270] [v2:127.0.0.1:6802/501244270,v1:127.0.0.1:6803/501244270] exists,up 05699e62-942f-4c78-b294-f0df41dceb2e 2026-02-12T19:14:13.152 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:14:13.152 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2323: test_get_op_scheduler: get_op_scheduler 0 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: get_op_scheduler: local id=0 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2311: get_op_scheduler: get_config osd 0 osd_op_queue 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=0 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.0 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-12T19:14:13.153 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:13.154 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:13.154 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:13.154 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-osd.0.asok 2026-02-12T19:14:13.154 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:13.154 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-osd.0.asok config get osd_op_queue 2026-02-12T19:14:13.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2323: test_get_op_scheduler: test wpq = wpq 2026-02-12T19:14:13.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2325: test_get_op_scheduler: run_osd td/ceph-helpers 1 --osd_op_queue=mclock_scheduler 2026-02-12T19:14:13.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:14:13.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:14:13.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T19:14:13.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:14:13.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:13.217 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:14:13.218 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:14:13.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:14:13.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_op_queue=mclock_scheduler 2026-02-12T19:14:13.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-12T19:14:13.219 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:14:13.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=09806aa7-db33-49f1-aa8c-c0cc2c118f69 2026-02-12T19:14:13.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 09806aa7-db33-49f1-aa8c-c0cc2c118f69' 2026-02-12T19:14:13.220 INFO:tasks.workunit.client.0.trial010.stdout:add osd1 09806aa7-db33-49f1-aa8c-c0cc2c118f69 2026-02-12T19:14:13.220 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:14:13.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCFJo5pxgVsDRAABvL8n4v1wNTW31pkGWYwwA== 2026-02-12T19:14:13.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCFJo5pxgVsDRAABvL8n4v1wNTW31pkGWYwwA=="}' 2026-02-12T19:14:13.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 09806aa7-db33-49f1-aa8c-c0cc2c118f69 -i td/ceph-helpers/1/new.json 2026-02-12T19:14:13.452 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:14:13.461 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-12T19:14:13.462 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler --mkfs --key AQCFJo5pxgVsDRAABvL8n4v1wNTW31pkGWYwwA== --osd-uuid 09806aa7-db33-49f1-aa8c-c0cc2c118f69 2026-02-12T19:14:13.479 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:13.472+0000 7f0eaf8d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:13.480 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:13.472+0000 7f0eaf8d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:13.481 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:13.476+0000 7f0eaf8d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:13.481 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:13.476+0000 7f0eaf8d6a40 -1 bdev(0x55834eb08400 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-12T19:14:13.481 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:13.476+0000 7f0eaf8d6a40 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-12T19:14:14.171 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-12T19:14:14.172 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:14:14.173 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T19:14:14.173 INFO:tasks.workunit.client.0.trial010.stdout:adding osd1 key to auth repository 2026-02-12T19:14:14.174 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:14:14.457 INFO:tasks.workunit.client.0.trial010.stdout:start osd.1 2026-02-12T19:14:14.458 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T19:14:14.458 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_op_queue=mclock_scheduler 2026-02-12T19:14:14.458 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:14:14.458 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:14:14.459 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:14:14.480 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:14.476+0000 7f32feeaca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:14.482 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:14.476+0000 7f32feeaca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:14.483 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:14.476+0000 7f32feeaca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:14.677 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T19:14:14.677 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:14:14.677 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T19:14:14.677 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:14:14.677 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:14:14.678 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:14.678 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:14.678 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:14:14.679 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:14.679 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:14:14.882 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:14.876+0000 7f32feeaca40 -1 Falling back to public interface 2026-02-12T19:14:14.890 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:15.493 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:15.488+0000 7f32feeaca40 -1 osd.1 0 log_to_monitors true 2026-02-12T19:14:15.891 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:15.891 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:15.891 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:14:15.892 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:14:15.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:15.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:14:16.101 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:17.102 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:17.103 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:17.103 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:14:17.104 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:14:17.104 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:17.104 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:14:17.321 INFO:tasks.workunit.client.0.trial010.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2215375387,v1:127.0.0.1:6811/2215375387] [v2:127.0.0.1:6812/2215375387,v1:127.0.0.1:6813/2215375387] exists,up 09806aa7-db33-49f1-aa8c-c0cc2c118f69 2026-02-12T19:14:17.323 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:14:17.323 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:14:17.323 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:14:17.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: test_get_op_scheduler: get_op_scheduler 1 2026-02-12T19:14:17.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2309: get_op_scheduler: local id=1 2026-02-12T19:14:17.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2311: get_op_scheduler: get_config osd 1 osd_op_queue 2026-02-12T19:14:17.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=osd 2026-02-12T19:14:17.324 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=1 2026-02-12T19:14:17.324 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=osd_op_queue 2026-02-12T19:14:17.324 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .osd_op_queue 2026-02-12T19:14:17.324 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path osd.1 2026-02-12T19:14:17.324 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-12T19:14:17.324 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-12T19:14:17.324 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:17.324 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:17.324 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:17.324 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-osd.1.asok 2026-02-12T19:14:17.325 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:17.325 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-osd.1.asok config get osd_op_queue 2026-02-12T19:14:17.384 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2326: test_get_op_scheduler: test mclock_scheduler = mclock_scheduler 2026-02-12T19:14:17.384 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2327: test_get_op_scheduler: teardown td/ceph-helpers 2026-02-12T19:14:17.384 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:17.384 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:17.384 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:17.385 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:17.385 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:17.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:17.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:17.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:17.492 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:17.492 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:17.493 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:17.493 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:17.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:17.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:17.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:17.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:17.497 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:17.497 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:17.497 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:17.498 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:17.499 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:17.499 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:17.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:17.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:17.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:17.505 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:17.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:17.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:17.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:14:17.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_osd_id_used_by_pgs td/ceph-helpers 2026-02-12T19:14:17.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1450: test_get_osd_id_used_by_pgs: local dir=td/ceph-helpers 2026-02-12T19:14:17.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1452: test_get_osd_id_used_by_pgs: setup td/ceph-helpers 2026-02-12T19:14:17.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:14:17.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:14:17.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:17.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:17.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:17.509 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:17.509 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:17.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:17.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:17.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:17.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:17.510 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:17.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:17.511 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:17.512 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:17.512 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:17.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:17.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:17.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:17.514 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:17.514 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:17.515 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:17.516 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:17.516 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:17.517 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:17.518 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:17.518 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:17.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:17.519 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:17.519 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:17.519 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:14:17.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:14:17.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:17.521 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:17.521 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:14:17.522 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:14:17.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:14:17.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:14:17.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:14:17.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1453: test_get_osd_id_used_by_pgs: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:17.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:14:17.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:14:17.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:14:17.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:14:17.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:14:17.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:17.545 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:14:17.545 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:17.545 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:17.546 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:17.546 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:17.546 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:17.546 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:17.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:17.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:14:17.577 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:14:17.577 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:17.578 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:17.578 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:14:17.579 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:14:17.579 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:17.579 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:17.579 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:17.579 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:17.579 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:17.580 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:17.580 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:17.580 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:17.580 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:14:17.636 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:14:17.636 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:17.636 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:17.636 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:14:17.636 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:14:17.637 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:17.637 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:17.637 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:17.638 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:17.638 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:17.638 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:17.638 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:17.638 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:17.638 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:14:17.692 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1454: test_get_osd_id_used_by_pgs: run_mgr td/ceph-helpers x 2026-02-12T19:14:17.692 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:14:17.693 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:14:17.693 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:14:17.693 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:14:17.693 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:14:17.693 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:14:17.806 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:14:17.807 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:17.807 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:17.807 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:17.807 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:17.807 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:17.808 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:17.808 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:17.809 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:17.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1455: test_get_osd_id_used_by_pgs: run_osd td/ceph-helpers 0 2026-02-12T19:14:17.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:14:17.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:14:17.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:14:17.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:14:17.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:17.830 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:14:17.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:14:17.832 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:14:17.832 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:14:17.832 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:14:17.832 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:14:17.832 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:14:17.833 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:14:17.833 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8082b52b-2f1c-4122-8f0f-f96331768afb 2026-02-12T19:14:17.833 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8082b52b-2f1c-4122-8f0f-f96331768afb' 2026-02-12T19:14:17.833 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 8082b52b-2f1c-4122-8f0f-f96331768afb 2026-02-12T19:14:17.833 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:14:17.842 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCJJo5pMYPpMRAA4nsFyqv2hez14gOKKrHA6A== 2026-02-12T19:14:17.842 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCJJo5pMYPpMRAA4nsFyqv2hez14gOKKrHA6A=="}' 2026-02-12T19:14:17.842 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8082b52b-2f1c-4122-8f0f-f96331768afb -i td/ceph-helpers/0/new.json 2026-02-12T19:14:17.952 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:17.959 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:14:17.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCJJo5pMYPpMRAA4nsFyqv2hez14gOKKrHA6A== --osd-uuid 8082b52b-2f1c-4122-8f0f-f96331768afb 2026-02-12T19:14:17.977 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:17.972+0000 7f4fe891ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:17.978 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:17.972+0000 7f4fe891ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:17.980 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:17.972+0000 7f4fe891ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:17.980 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:17.972+0000 7f4fe891ea40 -1 bdev(0x55a7e8cc2400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:14:17.980 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:17.972+0000 7f4fe891ea40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:14:18.756 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:14:18.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:14:18.757 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:14:18.758 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:14:18.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:14:18.880 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:14:18.880 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:14:18.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:14:18.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:14:18.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:14:18.881 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:14:18.906 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:18.900+0000 7f8fceaa6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:18.908 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:18.900+0000 7f8fceaa6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:18.909 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:18.904+0000 7f8fceaa6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:18.992 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:14:18.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:14:18.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:14:18.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:14:18.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:14:18.993 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:18.993 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:18.994 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:14:18.994 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:18.994 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:19.102 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:19.338 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:19.332+0000 7f8fceaa6a40 -1 Falling back to public interface 2026-02-12T19:14:20.103 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:20.103 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:20.104 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:14:20.104 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:14:20.104 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:20.105 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:20.213 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:20.398 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:20.392+0000 7f8fceaa6a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:14:21.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:21.215 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:21.215 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:14:21.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:14:21.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:21.216 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:21.430 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3063805177,v1:127.0.0.1:6801/3063805177] [v2:127.0.0.1:6802/3063805177,v1:127.0.0.1:6803/3063805177] exists,up 8082b52b-2f1c-4122-8f0f-f96331768afb 2026-02-12T19:14:21.431 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:14:21.431 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:14:21.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:14:21.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1456: test_get_osd_id_used_by_pgs: create_rbd_pool 2026-02-12T19:14:21.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:14:21.639 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:14:21.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:14:21.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:14:21.877 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:14:21.887 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:14:22.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:14:23.190 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1457: test_get_osd_id_used_by_pgs: wait_for_clean 2026-02-12T19:14:23.191 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:14:23.191 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:14:23.191 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:14:23.191 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:14:23.192 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:23.192 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:23.192 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:23.192 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:23.192 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:23.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:14:23.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:14:23.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:14:23.286 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:14:23.287 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:14:23.287 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:14:23.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:14:23.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:14:23.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:14:23.496 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:14:23.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-12T19:14:23.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-12T19:14:23.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-12T19:14:23.576 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:14:23.577 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-12T19:14:23.577 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:14:23.577 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:14:23.578 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-12T19:14:23.578 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:14:23.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-12T19:14:23.579 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-12T19:14:23.580 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836483 2026-02-12T19:14:23.580 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:14:23.799 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-12T19:14:23.799 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:14:24.800 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:14:24.801 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:14:25.020 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-12T19:14:25.020 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:14:25.021 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:25.021 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:25.314 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:14:25.314 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:14:25.315 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:14:25.315 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:14:25.315 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:14:25.315 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:14:25.315 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:14:25.315 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:14:25.528 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:14:25.529 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:14:25.529 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:25.529 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:25.819 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:14:25.820 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:14:25.820 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:14:25.821 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: get_osd_id_used_by_pgs 2026-02-12T19:14:25.821 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: uniq 2026-02-12T19:14:25.821 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: ceph --format json pg dump pgs 2026-02-12T19:14:25.821 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: jq '.pg_stats | .[] | .up[], .acting[]' 2026-02-12T19:14:25.821 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1446: get_osd_id_used_by_pgs: sort 2026-02-12T19:14:26.035 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1458: test_get_osd_id_used_by_pgs: local osd_ids=0 2026-02-12T19:14:26.035 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1459: test_get_osd_id_used_by_pgs: test 0 = 0 2026-02-12T19:14:26.035 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1460: test_get_osd_id_used_by_pgs: teardown td/ceph-helpers 2026-02-12T19:14:26.035 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:26.035 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:26.035 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:26.035 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:26.035 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:26.036 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:26.036 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:26.036 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:26.142 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:26.142 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:26.143 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:26.143 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:26.145 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:26.145 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:26.145 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:26.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:26.146 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:26.147 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:26.147 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:26.148 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:26.149 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:26.149 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:26.152 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:26.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:26.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:26.153 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:26.155 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:26.155 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:26.155 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:14:26.155 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_osds td/ceph-helpers 2026-02-12T19:14:26.155 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: test_get_osds: local dir=td/ceph-helpers 2026-02-12T19:14:26.155 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1024: test_get_osds: setup td/ceph-helpers 2026-02-12T19:14:26.156 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:14:26.156 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:14:26.156 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:26.156 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:26.156 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:26.156 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:26.156 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:26.156 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:26.156 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:26.156 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:26.157 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:26.157 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:26.158 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:26.159 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:26.160 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:26.160 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:26.160 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:26.161 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:26.161 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:26.162 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:26.162 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:26.163 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:26.164 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:26.164 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:26.165 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:26.165 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:26.165 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:26.165 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:26.166 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:26.166 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:26.167 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:14:26.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:14:26.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:26.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:26.168 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:14:26.170 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:14:26.171 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:14:26.171 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:14:26.171 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:14:26.171 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: test_get_osds: run_mon td/ceph-helpers a --osd_pool_default_size=2 2026-02-12T19:14:26.171 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:14:26.171 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:14:26.171 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:14:26.171 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:14:26.172 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:14:26.172 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=2 2026-02-12T19:14:26.193 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:14:26.193 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:26.193 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:26.193 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:26.193 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:26.193 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:26.194 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:26.194 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=2 2026-02-12T19:14:26.223 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:14:26.224 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:14:26.224 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:26.224 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:26.224 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:14:26.225 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:14:26.225 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:26.226 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:26.226 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:26.226 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:26.226 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:26.226 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:26.226 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:26.226 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:26.226 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:14:26.281 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:14:26.281 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:26.281 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:26.281 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:14:26.281 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:14:26.283 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:26.283 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:26.283 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:26.283 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:26.283 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:26.283 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:26.283 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:26.283 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:26.283 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:14:26.337 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1026: test_get_osds: run_mgr td/ceph-helpers x 2026-02-12T19:14:26.337 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:14:26.337 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:14:26.337 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:14:26.337 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:14:26.337 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:14:26.337 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:14:26.447 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:14:26.448 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:26.448 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:26.448 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:26.448 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:26.448 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:26.448 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:26.448 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:26.449 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:26.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: test_get_osds: run_osd td/ceph-helpers 0 2026-02-12T19:14:26.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:14:26.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:14:26.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:26.469 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:14:26.470 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:14:26.472 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:14:26.473 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b886a93a-ff25-4790-ad49-261a117a29f7 2026-02-12T19:14:26.473 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b886a93a-ff25-4790-ad49-261a117a29f7' 2026-02-12T19:14:26.473 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 b886a93a-ff25-4790-ad49-261a117a29f7 2026-02-12T19:14:26.473 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:14:26.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCSJo5pty2VHBAAV14qWzMz5AQP4JwQMdjyIQ== 2026-02-12T19:14:26.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCSJo5pty2VHBAAV14qWzMz5AQP4JwQMdjyIQ=="}' 2026-02-12T19:14:26.484 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b886a93a-ff25-4790-ad49-261a117a29f7 -i td/ceph-helpers/0/new.json 2026-02-12T19:14:26.601 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:26.609 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:14:26.610 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCSJo5pty2VHBAAV14qWzMz5AQP4JwQMdjyIQ== --osd-uuid b886a93a-ff25-4790-ad49-261a117a29f7 2026-02-12T19:14:26.626 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:26.620+0000 7fa83d8d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:26.628 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:26.620+0000 7fa83d8d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:26.629 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:26.624+0000 7fa83d8d1a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:26.629 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:26.624+0000 7fa83d8d1a40 -1 bdev(0x56227cbee400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:14:26.630 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:26.624+0000 7fa83d8d1a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:14:27.400 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:14:27.401 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:14:27.402 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:14:27.402 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:14:27.402 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:14:27.522 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:14:27.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:14:27.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:14:27.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:14:27.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:14:27.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:14:27.547 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:27.540+0000 7fe77741da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:27.549 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:27.544+0000 7fe77741da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:27.550 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:27.544+0000 7fe77741da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:27.630 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:14:27.630 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:14:27.630 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:14:27.630 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:14:27.630 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:14:27.630 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:27.630 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:27.631 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:14:27.631 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:27.631 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:27.737 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:27.990 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:27.984+0000 7fe77741da40 -1 Falling back to public interface 2026-02-12T19:14:28.738 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:28.738 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:28.739 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:14:28.739 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:14:28.739 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:28.740 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:28.822 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:28.816+0000 7fe77741da40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:14:28.843 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:29.845 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:29.845 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:29.846 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:14:29.847 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:14:29.847 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:29.847 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:30.053 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3070733892,v1:127.0.0.1:6801/3070733892] [v2:127.0.0.1:6802/3070733892,v1:127.0.0.1:6803/3070733892] exists,up b886a93a-ff25-4790-ad49-261a117a29f7 2026-02-12T19:14:30.056 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:14:30.056 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:14:30.056 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:14:30.056 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1028: test_get_osds: run_osd td/ceph-helpers 1 2026-02-12T19:14:30.056 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/1 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/1' 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/1/journal' 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:14:30.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:14:30.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:14:30.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:14:30.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:14:30.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:14:30.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:14:30.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:14:30.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:14:30.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/1 2026-02-12T19:14:30.060 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:14:30.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5d75a325-0ec7-4427-8dde-d08e9ea5a85e 2026-02-12T19:14:30.060 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 5d75a325-0ec7-4427-8dde-d08e9ea5a85e' 2026-02-12T19:14:30.060 INFO:tasks.workunit.client.0.trial010.stdout:add osd1 5d75a325-0ec7-4427-8dde-d08e9ea5a85e 2026-02-12T19:14:30.061 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:14:30.069 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCWJo5p2wXXAxAATFrH+RMNfHsxL7U7C1W3LA== 2026-02-12T19:14:30.069 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCWJo5p2wXXAxAATFrH+RMNfHsxL7U7C1W3LA=="}' 2026-02-12T19:14:30.069 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5d75a325-0ec7-4427-8dde-d08e9ea5a85e -i td/ceph-helpers/1/new.json 2026-02-12T19:14:30.289 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:14:30.298 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/1/new.json 2026-02-12T19:14:30.299 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCWJo5p2wXXAxAATFrH+RMNfHsxL7U7C1W3LA== --osd-uuid 5d75a325-0ec7-4427-8dde-d08e9ea5a85e 2026-02-12T19:14:30.315 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:30.308+0000 7f57ddc01a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:30.317 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:30.312+0000 7f57ddc01a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:30.318 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:30.312+0000 7f57ddc01a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:30.319 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:30.312+0000 7f57ddc01a40 -1 bdev(0x55caae6ba400 td/ceph-helpers/1/block) open stat got: (1) Operation not permitted 2026-02-12T19:14:30.319 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:30.312+0000 7f57ddc01a40 -1 bluestore(td/ceph-helpers/1) _read_fsid unparsable uuid 2026-02-12T19:14:30.951 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/1/keyring 2026-02-12T19:14:30.951 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:14:30.952 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-12T19:14:30.953 INFO:tasks.workunit.client.0.trial010.stdout:adding osd1 key to auth repository 2026-02-12T19:14:30.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:14:31.246 INFO:tasks.workunit.client.0.trial010.stdout:start osd.1 2026-02-12T19:14:31.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-12T19:14:31.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/1 --osd-journal=td/ceph-helpers/1/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:14:31.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:14:31.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:14:31.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:14:31.262 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:31.256+0000 7fe936957a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:31.264 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:31.256+0000 7fe936957a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:31.265 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:31.260+0000 7fe936957a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:31.458 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-12T19:14:31.458 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:14:31.458 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-12T19:14:31.458 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:14:31.459 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:14:31.459 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:31.459 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:14:31.459 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:31.459 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:31.460 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:14:31.662 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:31.656+0000 7fe936957a40 -1 Falling back to public interface 2026-02-12T19:14:31.679 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:32.239 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:32.232+0000 7fe936957a40 -1 osd.1 0 log_to_monitors true 2026-02-12T19:14:32.680 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:32.681 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:32.681 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:14:32.682 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:14:32.682 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:32.682 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-12T19:14:32.895 INFO:tasks.workunit.client.0.trial010.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1805179031,v1:127.0.0.1:6811/1805179031] [v2:127.0.0.1:6812/1805179031,v1:127.0.0.1:6813/1805179031] exists,up 5d75a325-0ec7-4427-8dde-d08e9ea5a85e 2026-02-12T19:14:32.895 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:14:32.896 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:14:32.896 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:14:32.896 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1029: test_get_osds: create_rbd_pool 2026-02-12T19:14:32.896 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:14:33.104 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:14:33.115 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:14:33.115 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:14:33.375 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:14:33.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:14:34.387 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:14:34.669 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1030: test_get_osds: wait_for_clean 2026-02-12T19:14:34.669 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:14:34.669 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:14:34.669 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:14:34.669 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:14:34.670 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:34.670 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:34.670 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:34.670 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:34.670 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:34.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:14:34.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:14:34.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:14:34.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:14:34.767 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:14:34.767 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:14:34.986 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-12T19:14:34.986 INFO:tasks.workunit.client.0.trial010.stderr:1' 2026-02-12T19:14:34.986 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:14:34.986 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:14:34.986 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:14:35.069 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-12T19:14:35.069 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-12T19:14:35.069 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484' 2026-02-12T19:14:35.069 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:14:35.069 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.1 flush_pg_stats 2026-02-12T19:14:35.150 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-12T19:14:35.150 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-12T19:14:35.150 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836484 1-42949672964' 2026-02-12T19:14:35.150 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:14:35.151 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836484 2026-02-12T19:14:35.151 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:14:35.152 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:14:35.152 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-12T19:14:35.153 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:14:35.154 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836484 2026-02-12T19:14:35.154 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-12T19:14:35.154 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836484 2026-02-12T19:14:35.155 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:14:35.381 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836484 2026-02-12T19:14:35.381 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:14:36.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:14:36.383 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:14:36.596 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836484 2026-02-12T19:14:36.596 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:14:37.597 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T19:14:37.598 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:14:37.816 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-12T19:14:37.816 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:14:37.816 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 1-42949672964 2026-02-12T19:14:37.816 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:14:37.818 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=1 2026-02-12T19:14:37.818 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-12T19:14:37.818 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:14:37.820 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=42949672964 2026-02-12T19:14:37.820 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.1 seq 42949672964 2026-02-12T19:14:37.820 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-12T19:14:37.821 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-12T19:14:38.046 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-12T19:14:38.046 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:14:38.047 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:38.047 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:38.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:14:38.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:14:38.332 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:14:38.332 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:14:38.332 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:14:38.332 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:14:38.332 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:14:38.332 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:14:38.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:14:38.547 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:14:38.547 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:38.547 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:38.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:14:38.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:14:38.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:14:38.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1031: test_get_osds: create_rbd_pool 2026-02-12T19:14:38.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:14:39.085 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:14:39.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:14:39.096 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:14:39.355 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:14:39.365 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:14:40.366 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:14:40.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: get_osds rbd GROUP 2026-02-12T19:14:40.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=rbd 2026-02-12T19:14:40.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=GROUP 2026-02-12T19:14:40.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1032: test_get_osds: grep --quiet '^[0-1] [0-1]$' 2026-02-12T19:14:40.649 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map rbd GROUP 2026-02-12T19:14:40.649 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-12T19:14:40.866 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-12T19:14:40.866 INFO:tasks.workunit.client.0.trial010.stderr:0' 2026-02-12T19:14:40.867 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2026-02-12T19:14:40.867 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1033: test_get_osds: teardown td/ceph-helpers 2026-02-12T19:14:40.867 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:40.867 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:40.867 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:40.868 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:40.868 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:40.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:40.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:40.868 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:41.177 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:41.177 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:41.178 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:41.179 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:41.181 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:41.181 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:41.181 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:41.182 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:41.183 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:41.183 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:41.183 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:41.184 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:41.185 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:41.185 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:41.192 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:41.192 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:41.192 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:41.192 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:41.194 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:41.194 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_pg td/ceph-helpers 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1091: test_get_pg: local dir=td/ceph-helpers 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: test_get_pg: setup td/ceph-helpers 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:41.195 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:41.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:41.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:41.196 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:41.197 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:41.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:41.198 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:41.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:41.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:41.199 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:41.201 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:41.201 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:41.201 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:41.201 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:41.202 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:41.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:41.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:41.205 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:41.205 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:41.205 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:41.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:41.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:41.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:41.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:14:41.207 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:14:41.207 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:41.207 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:41.207 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:14:41.209 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:14:41.210 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:14:41.210 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:14:41.210 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:14:41.210 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: test_get_pg: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:41.210 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:14:41.210 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:14:41.211 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:14:41.211 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:14:41.211 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:14:41.211 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:41.232 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:14:41.233 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:41.233 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:41.233 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:41.233 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:41.233 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:41.233 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:41.234 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:41.268 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:14:41.269 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:14:41.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:41.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:41.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:14:41.271 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:14:41.271 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:41.271 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:41.271 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:41.271 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:41.271 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:41.271 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:41.271 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:41.272 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:41.272 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:14:41.327 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:14:41.328 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:41.328 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:41.328 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:14:41.328 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:14:41.329 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:41.329 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:41.329 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:41.329 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:41.330 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:41.330 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:41.330 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:41.330 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:41.330 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:14:41.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1095: test_get_pg: run_mgr td/ceph-helpers x 2026-02-12T19:14:41.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:14:41.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:14:41.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:14:41.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:14:41.386 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:14:41.386 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:14:41.501 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:14:41.501 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:41.501 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:41.501 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:41.502 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:41.502 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:41.502 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:41.502 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:41.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:41.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: test_get_pg: run_osd td/ceph-helpers 0 2026-02-12T19:14:41.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:14:41.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:14:41.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:14:41.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:14:41.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:14:41.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:14:41.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:14:41.524 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:41.525 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:14:41.526 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:14:41.527 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:14:41.527 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:14:41.527 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9451c8e8-65d7-4e89-9b01-9e8778bfba75 2026-02-12T19:14:41.528 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 9451c8e8-65d7-4e89-9b01-9e8778bfba75 2026-02-12T19:14:41.528 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 9451c8e8-65d7-4e89-9b01-9e8778bfba75' 2026-02-12T19:14:41.528 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:14:41.538 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQChJo5pLoLLHxAAzelOPOVx7ejUnU6Bl2j/Ow== 2026-02-12T19:14:41.538 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQChJo5pLoLLHxAAzelOPOVx7ejUnU6Bl2j/Ow=="}' 2026-02-12T19:14:41.538 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9451c8e8-65d7-4e89-9b01-9e8778bfba75 -i td/ceph-helpers/0/new.json 2026-02-12T19:14:41.651 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:41.659 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:14:41.660 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQChJo5pLoLLHxAAzelOPOVx7ejUnU6Bl2j/Ow== --osd-uuid 9451c8e8-65d7-4e89-9b01-9e8778bfba75 2026-02-12T19:14:41.675 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:41.668+0000 7fa9bf56da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:41.677 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:41.672+0000 7fa9bf56da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:41.678 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:41.672+0000 7fa9bf56da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:41.678 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:41.672+0000 7fa9bf56da40 -1 bdev(0x55c339670400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:14:41.679 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:41.672+0000 7fa9bf56da40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:14:42.401 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:14:42.401 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:14:42.402 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:14:42.402 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:14:42.403 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:14:42.522 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:14:42.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:14:42.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:14:42.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:14:42.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:14:42.523 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:14:42.546 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:42.540+0000 7f7ad439fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:42.548 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:42.540+0000 7f7ad439fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:42.549 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:42.544+0000 7f7ad439fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:42.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:14:42.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:14:42.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:14:42.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:14:42.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:14:42.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:42.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:14:42.633 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:42.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:42.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:42.738 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:43.090 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:43.084+0000 7f7ad439fa40 -1 Falling back to public interface 2026-02-12T19:14:43.739 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:43.739 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:43.740 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:14:43.740 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:14:43.740 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:43.741 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:43.845 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:44.011 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:44.004+0000 7f7ad439fa40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:14:44.847 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:44.847 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:44.847 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:14:44.847 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:14:44.847 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:44.847 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:45.061 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2614809487,v1:127.0.0.1:6801/2614809487] [v2:127.0.0.1:6802/2614809487,v1:127.0.0.1:6803/2614809487] exists,up 9451c8e8-65d7-4e89-9b01-9e8778bfba75 2026-02-12T19:14:45.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:14:45.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:14:45.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:14:45.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1097: test_get_pg: create_rbd_pool 2026-02-12T19:14:45.061 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:14:45.262 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:14:45.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:14:45.272 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:14:45.506 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:14:45.516 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:14:46.517 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:14:46.795 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1098: test_get_pg: wait_for_clean 2026-02-12T19:14:46.795 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:14:46.795 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:14:46.795 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:14:46.795 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:14:46.796 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:46.796 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:46.796 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:46.796 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:46.796 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:46.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:14:46.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:14:46.893 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:14:46.893 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:14:46.893 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:14:46.893 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:14:47.104 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:14:47.104 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:14:47.104 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:14:47.104 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:14:47.184 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-12T19:14:47.184 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-12T19:14:47.184 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-12T19:14:47.184 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:14:47.185 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-12T19:14:47.185 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:14:47.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:14:47.186 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-12T19:14:47.187 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:14:47.188 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-12T19:14:47.188 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-12T19:14:47.188 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836483 2026-02-12T19:14:47.188 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:14:47.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-12T19:14:47.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:14:48.415 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:14:48.416 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:14:48.635 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-12T19:14:48.635 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:14:48.636 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:48.636 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:48.931 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:14:48.932 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:14:48.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:14:48.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:14:48.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:14:48.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:14:48.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:14:48.932 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:14:49.144 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:14:49.145 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:14:49.145 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:49.145 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:49.436 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:14:49.436 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:14:49.436 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:14:49.436 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: get_pg rbd GROUP 2026-02-12T19:14:49.436 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=rbd 2026-02-12T19:14:49.436 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=GROUP 2026-02-12T19:14:49.436 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1099: test_get_pg: grep --quiet '^[0-9]\.[0-9a-f][0-9a-f]*$' 2026-02-12T19:14:49.436 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map rbd GROUP 2026-02-12T19:14:49.437 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-12T19:14:49.650 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1100: test_get_pg: teardown td/ceph-helpers 2026-02-12T19:14:49.650 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:49.651 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:49.651 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:49.651 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:49.651 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:49.651 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:49.651 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:49.651 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:49.758 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:49.759 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:49.760 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:49.760 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:49.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:49.761 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:49.762 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:49.763 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:49.763 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:49.763 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:49.764 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:49.765 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:49.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:49.766 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:49.771 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:49.772 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:49.772 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:49.772 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:49.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:49.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:49.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:14:49.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_primary td/ceph-helpers 2026-02-12T19:14:49.774 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1195: test_get_primary: local dir=td/ceph-helpers 2026-02-12T19:14:49.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1197: test_get_primary: setup td/ceph-helpers 2026-02-12T19:14:49.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:14:49.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:14:49.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:49.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:49.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:49.775 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:49.775 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:49.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:49.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:49.775 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:49.776 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:49.776 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:49.777 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:49.778 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:49.778 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:49.779 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:49.779 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:49.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:49.780 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:49.781 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:49.781 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:49.781 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:49.782 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:49.782 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:49.784 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:49.784 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:49.784 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:49.784 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:49.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:49.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:49.785 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:14:49.786 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:14:49.786 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:49.786 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:49.787 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:14:49.788 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:14:49.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:14:49.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:14:49.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:14:49.789 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1198: test_get_primary: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:49.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:14:49.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:14:49.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:14:49.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:14:49.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:14:49.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:49.809 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:14:49.810 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:49.810 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:49.810 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:49.810 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:49.810 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:49.810 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:49.810 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:49.839 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:14:49.840 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:14:49.840 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:49.840 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:49.840 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:14:49.841 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:14:49.841 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:49.841 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:49.842 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:49.842 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:49.842 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:49.842 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:49.842 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:49.842 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:49.842 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:14:49.896 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:14:49.896 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:49.896 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:49.896 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:14:49.896 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:14:49.898 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:49.898 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:49.898 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:49.898 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:49.898 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:49.898 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:49.898 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:49.898 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:49.898 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:14:49.951 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1199: test_get_primary: local osd=0 2026-02-12T19:14:49.951 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1200: test_get_primary: run_mgr td/ceph-helpers x 2026-02-12T19:14:49.951 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:14:49.951 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:14:49.951 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:14:49.952 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:14:49.952 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:14:49.952 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:14:50.068 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:14:50.068 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:50.068 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:50.068 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:50.068 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:50.068 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:50.068 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:50.069 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:50.069 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:50.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1201: test_get_primary: run_osd td/ceph-helpers 0 2026-02-12T19:14:50.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:14:50.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:14:50.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:14:50.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:14:50.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:14:50.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:14:50.089 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:50.090 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:14:50.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:14:50.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:14:50.092 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:14:50.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2263d34a-cbf4-45b3-a9ca-de56ab2b9dc0 2026-02-12T19:14:50.092 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 2263d34a-cbf4-45b3-a9ca-de56ab2b9dc0 2026-02-12T19:14:50.093 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2263d34a-cbf4-45b3-a9ca-de56ab2b9dc0' 2026-02-12T19:14:50.093 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:14:50.102 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCqJo5pR9LQBRAAVCCL3CmSXraYpcOlCFTXWg== 2026-02-12T19:14:50.102 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCqJo5pR9LQBRAAVCCL3CmSXraYpcOlCFTXWg=="}' 2026-02-12T19:14:50.102 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2263d34a-cbf4-45b3-a9ca-de56ab2b9dc0 -i td/ceph-helpers/0/new.json 2026-02-12T19:14:50.216 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:50.225 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:14:50.226 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCqJo5pR9LQBRAAVCCL3CmSXraYpcOlCFTXWg== --osd-uuid 2263d34a-cbf4-45b3-a9ca-de56ab2b9dc0 2026-02-12T19:14:50.248 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:50.240+0000 7f867b775a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:50.249 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:50.244+0000 7f867b775a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:50.251 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:50.244+0000 7f867b775a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:50.251 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:50.244+0000 7f867b775a40 -1 bdev(0x56471ca5a400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:14:50.251 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:50.244+0000 7f867b775a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:14:51.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:14:51.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:14:51.137 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:14:51.138 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:14:51.138 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:14:51.250 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:14:51.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:14:51.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:14:51.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:14:51.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:14:51.251 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:14:51.265 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:51.259+0000 7f72a5b7ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:51.267 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:51.259+0000 7f72a5b7ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:51.268 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:51.259+0000 7f72a5b7ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:51.364 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:14:51.364 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:14:51.364 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:14:51.364 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:14:51.364 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:14:51.364 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:51.365 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:14:51.365 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:51.365 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:51.365 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:51.476 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:51.826 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:51.819+0000 7f72a5b7ca40 -1 Falling back to public interface 2026-02-12T19:14:52.477 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:52.477 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:52.478 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:14:52.478 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:14:52.478 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:52.479 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:52.694 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:14:52.729 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:52.723+0000 7f72a5b7ca40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:14:53.696 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:14:53.696 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:14:53.696 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:14:53.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:14:53.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:14:53.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:14:53.915 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3393140119,v1:127.0.0.1:6801/3393140119] [v2:127.0.0.1:6802/3393140119,v1:127.0.0.1:6803/3393140119] exists,up 2263d34a-cbf4-45b3-a9ca-de56ab2b9dc0 2026-02-12T19:14:53.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:14:53.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:14:53.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:14:53.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1202: test_get_primary: create_rbd_pool 2026-02-12T19:14:53.916 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:14:54.124 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:14:54.133 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:14:54.133 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:14:54.411 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:14:54.420 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:14:55.421 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:14:55.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1203: test_get_primary: wait_for_clean 2026-02-12T19:14:55.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:14:55.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:14:55.702 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:14:55.703 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:14:55.703 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:55.703 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:55.704 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:55.704 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:55.704 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:55.814 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:14:55.814 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:14:55.814 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:14:55.814 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:14:55.814 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:14:55.814 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:14:56.029 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:14:56.029 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:14:56.029 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:14:56.029 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:14:56.110 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-12T19:14:56.110 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-12T19:14:56.110 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-12T19:14:56.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:14:56.111 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-12T19:14:56.111 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:14:56.111 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:14:56.112 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-12T19:14:56.112 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:14:56.113 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-12T19:14:56.114 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836483 2026-02-12T19:14:56.114 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-12T19:14:56.114 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:14:56.333 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-12T19:14:56.334 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:14:57.335 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:14:57.335 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:14:57.554 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-12T19:14:57.555 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:14:57.555 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:57.555 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:57.844 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:14:57.845 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:14:57.845 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:14:57.845 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:14:57.845 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:14:57.845 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:14:57.845 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:14:57.846 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:14:58.059 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:14:58.059 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:14:58.060 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:14:58.060 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:14:58.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:14:58.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:14:58.347 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:14:58.347 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: get_primary rbd GROUP 2026-02-12T19:14:58.347 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=rbd 2026-02-12T19:14:58.347 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=GROUP 2026-02-12T19:14:58.348 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map rbd GROUP 2026-02-12T19:14:58.348 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-12T19:14:58.566 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1204: test_get_primary: test 0 = 0 2026-02-12T19:14:58.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1205: test_get_primary: teardown td/ceph-helpers 2026-02-12T19:14:58.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:58.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:58.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:58.567 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:58.567 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:58.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:58.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:58.567 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:58.674 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:58.675 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:58.676 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:58.676 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:58.677 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:58.677 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:58.678 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:58.679 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:58.679 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:58.680 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:58.680 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:58.681 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:58.682 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:58.682 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:58.687 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:58.687 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:58.688 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:58.688 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:58.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:58.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:58.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:14:58.689 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_get_timeout_delays td/ceph-helpers 2026-02-12T19:14:58.690 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: get_timeout_delays 1 2026-02-12T19:14:58.690 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:58.690 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:58.690 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:58.690 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:58.691 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:58.703 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1621: test_get_timeout_delays: test '1 ' = '1 ' 2026-02-12T19:14:58.703 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: get_timeout_delays 5 2026-02-12T19:14:58.704 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:58.704 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:58.705 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:58.705 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:58.705 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:58.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1622: test_get_timeout_delays: test '1 2 2 ' = '1 2 2 ' 2026-02-12T19:14:58.729 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: get_timeout_delays 6 2026-02-12T19:14:58.730 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:58.730 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:58.731 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:58.731 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:58.731 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:58.750 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1623: test_get_timeout_delays: test '1 2 3 ' = '1 2 3 ' 2026-02-12T19:14:58.750 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: get_timeout_delays 7 2026-02-12T19:14:58.751 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:58.751 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:58.752 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:58.752 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:58.752 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:58.781 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1624: test_get_timeout_delays: test '1 2 4 ' = '1 2 4 ' 2026-02-12T19:14:58.782 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: get_timeout_delays 8 2026-02-12T19:14:58.782 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:58.782 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:58.783 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:58.783 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:58.783 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:58.816 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1625: test_get_timeout_delays: test '1 2 4 1 ' = '1 2 4 1 ' 2026-02-12T19:14:58.816 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: get_timeout_delays 1 .1 2026-02-12T19:14:58.817 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:58.817 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:58.818 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:58.818 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:58.818 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:58.851 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1626: test_get_timeout_delays: test '0.1 0.2 0.4 0.3 ' = '0.1 0.2 0.4 0.3 ' 2026-02-12T19:14:58.851 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: get_timeout_delays 1.5 .1 2026-02-12T19:14:58.852 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:58.852 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:58.853 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:58.853 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:58.853 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:58.892 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1627: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 ' = '0.1 0.2 0.4 0.8 ' 2026-02-12T19:14:58.892 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: get_timeout_delays 5 .1 2026-02-12T19:14:58.893 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:58.893 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:58.894 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:58.894 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:58.894 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:58.944 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1628: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 1.9 ' = '0.1 0.2 0.4 0.8 1.6 1.9 ' 2026-02-12T19:14:58.944 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: get_timeout_delays 6 .1 2026-02-12T19:14:58.945 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:58.945 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:58.946 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:58.946 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:58.946 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:58.986 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1629: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 2.9 ' = '0.1 0.2 0.4 0.8 1.6 2.9 ' 2026-02-12T19:14:58.987 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: get_timeout_delays 6.3 .1 2026-02-12T19:14:58.988 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:58.988 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:58.988 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:58.988 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:58.988 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:59.036 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1630: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 ' = '0.1 0.2 0.4 0.8 1.6 3.2 ' 2026-02-12T19:14:59.037 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: get_timeout_delays 20 .1 2026-02-12T19:14:59.037 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:59.037 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:59.038 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:59.038 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:59.038 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:59.105 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1631: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 7.3 ' 2026-02-12T19:14:59.106 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: get_timeout_delays 300 .1 0 2026-02-12T19:14:59.106 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:59.106 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:59.107 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:59.107 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:59.107 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:59.191 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1632: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 12.8 25.6 51.2 102.4 95.3 ' 2026-02-12T19:14:59.191 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: get_timeout_delays 300 .1 10 2026-02-12T19:14:59.192 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:14:59.192 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:14:59.192 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:14:59.192 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:14:59.192 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:14:59.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1633: test_get_timeout_delays: test '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' = '0.1 0.2 0.4 0.8 1.6 3.2 6.4 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 7.3 ' 2026-02-12T19:14:59.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:14:59.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_is_clean td/ceph-helpers 2026-02-12T19:14:59.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1552: test_is_clean: local dir=td/ceph-helpers 2026-02-12T19:14:59.502 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1554: test_is_clean: setup td/ceph-helpers 2026-02-12T19:14:59.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:14:59.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:14:59.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:14:59.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:14:59.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:14:59.503 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:14:59.503 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:14:59.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:14:59.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:14:59.503 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:14:59.504 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:14:59.504 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:14:59.505 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:14:59.505 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:14:59.506 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:14:59.506 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:14:59.507 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:14:59.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:59.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:14:59.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:14:59.508 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:14:59.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:14:59.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:14:59.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:14:59.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:14:59.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:59.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:59.512 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:14:59.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:14:59.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:14:59.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:14:59.515 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:14:59.515 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:59.515 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:59.515 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:14:59.516 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:14:59.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:14:59.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:14:59.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:14:59.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1555: test_is_clean: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:59.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:14:59.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:14:59.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:14:59.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:14:59.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:14:59.518 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:59.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:14:59.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:59.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:59.540 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:59.540 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:59.541 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:59.541 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:59.541 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:14:59.569 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:14:59.570 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:14:59.570 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:59.571 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:59.571 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:14:59.572 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:14:59.572 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:59.572 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:59.572 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:59.572 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:59.573 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:59.573 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:59.573 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:59.573 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:59.573 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:14:59.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:14:59.629 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:14:59.629 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:14:59.629 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:14:59.629 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:14:59.630 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:14:59.630 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:14:59.630 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:14:59.630 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:14:59.630 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:59.631 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:59.631 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:14:59.631 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:14:59.631 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:14:59.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1556: test_is_clean: run_mgr td/ceph-helpers x 2026-02-12T19:14:59.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:14:59.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:14:59.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:14:59.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:14:59.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:14:59.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:14:59.804 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:14:59.804 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:59.804 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:59.804 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:59.804 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:59.804 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:59.805 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:59.805 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:59.806 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:14:59.827 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1557: test_is_clean: run_osd td/ceph-helpers 0 2026-02-12T19:14:59.827 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:14:59.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:14:59.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:14:59.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:14:59.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:14:59.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:14:59.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:14:59.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:14:59.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:14:59.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:14:59.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:14:59.828 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:14:59.829 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:14:59.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:14:59.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:14:59.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:14:59.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:14:59.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:14:59.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:14:59.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:14:59.830 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:14:59.831 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:14:59.831 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1b874790-ed9d-4696-98b0-f5507759aba0 2026-02-12T19:14:59.831 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 1b874790-ed9d-4696-98b0-f5507759aba0 2026-02-12T19:14:59.832 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1b874790-ed9d-4696-98b0-f5507759aba0' 2026-02-12T19:14:59.832 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:14:59.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCzJo5pbZXLMRAALjyrYxDnEiFdy3BLUAgnEA== 2026-02-12T19:14:59.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCzJo5pbZXLMRAALjyrYxDnEiFdy3BLUAgnEA=="}' 2026-02-12T19:14:59.840 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1b874790-ed9d-4696-98b0-f5507759aba0 -i td/ceph-helpers/0/new.json 2026-02-12T19:14:59.952 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:14:59.959 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:14:59.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCzJo5pbZXLMRAALjyrYxDnEiFdy3BLUAgnEA== --osd-uuid 1b874790-ed9d-4696-98b0-f5507759aba0 2026-02-12T19:14:59.977 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:59.971+0000 7f8fb8aa4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:59.979 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:59.971+0000 7f8fb8aa4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:59.980 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:59.971+0000 7f8fb8aa4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:14:59.980 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:59.971+0000 7f8fb8aa4a40 -1 bdev(0x559814040400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:14:59.980 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:14:59.971+0000 7f8fb8aa4a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:15:00.713 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:15:00.714 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:15:00.714 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:15:00.715 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:15:00.715 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:15:00.832 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:15:00.833 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:15:00.833 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:15:00.833 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:15:00.834 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:15:00.834 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:15:00.857 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:00.851+0000 7f0a5cf81a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:00.859 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:00.851+0000 7f0a5cf81a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:00.861 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:00.855+0000 7f0a5cf81a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:00.944 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:15:00.944 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:15:00.944 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:15:00.944 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:15:00.944 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:15:00.944 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:00.945 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:15:00.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:15:00.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:00.945 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:01.055 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:15:01.390 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:01.383+0000 7f0a5cf81a40 -1 Falling back to public interface 2026-02-12T19:15:02.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:15:02.057 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:02.057 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:15:02.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:15:02.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:02.058 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:02.167 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:15:02.378 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:02.371+0000 7f0a5cf81a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:15:03.168 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:15:03.168 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:03.169 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:15:03.169 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:15:03.170 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:03.170 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:03.382 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2716738808,v1:127.0.0.1:6801/2716738808] [v2:127.0.0.1:6802/2716738808,v1:127.0.0.1:6803/2716738808] exists,up 1b874790-ed9d-4696-98b0-f5507759aba0 2026-02-12T19:15:03.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:15:03.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:15:03.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:15:03.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1558: test_is_clean: create_rbd_pool 2026-02-12T19:15:03.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:15:03.591 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:15:03.601 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:15:03.601 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:15:03.828 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:15:03.837 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:15:04.838 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:15:05.115 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1559: test_is_clean: wait_for_clean 2026-02-12T19:15:05.115 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:15:05.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:15:05.116 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:15:05.116 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:15:05.117 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:15:05.117 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:15:05.117 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:15:05.117 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:15:05.117 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:15:05.213 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:15:05.213 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:15:05.213 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:15:05.213 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:15:05.213 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:15:05.213 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:15:05.427 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:15:05.427 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:15:05.427 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:15:05.428 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:15:05.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-12T19:15:05.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-12T19:15:05.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-12T19:15:05.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:15:05.510 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-12T19:15:05.510 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:15:05.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:15:05.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-12T19:15:05.512 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:15:05.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-12T19:15:05.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-12T19:15:05.513 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836483 2026-02-12T19:15:05.514 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:05.723 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-12T19:15:05.723 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:15:06.724 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:15:06.725 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:06.939 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-12T19:15:06.940 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:15:06.940 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:15:06.940 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:15:07.212 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:15:07.213 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:15:07.213 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:15:07.213 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:15:07.213 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:15:07.213 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:15:07.213 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:15:07.213 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:15:07.417 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:15:07.418 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:15:07.418 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:15:07.418 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:15:07.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:15:07.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:15:07.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:15:07.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1560: test_is_clean: is_clean 2026-02-12T19:15:07.698 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: get_num_pgs 2026-02-12T19:15:07.699 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:15:07.699 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:15:07.991 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1546: is_clean: num_pgs=4 2026-02-12T19:15:07.992 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1547: is_clean: test 4 '!=' 0 2026-02-12T19:15:07.992 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: get_num_active_clean 2026-02-12T19:15:07.992 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:15:07.992 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:15:07.992 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:15:07.993 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:15:07.993 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:15:08.208 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1548: is_clean: test 4 = 4 2026-02-12T19:15:08.208 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1561: test_is_clean: teardown td/ceph-helpers 2026-02-12T19:15:08.208 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:15:08.209 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:15:08.209 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:15:08.209 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:08.209 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:08.209 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:08.209 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:08.209 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:08.316 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:08.316 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:15:08.317 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:15:08.317 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:15:08.319 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:15:08.319 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:15:08.319 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:15:08.320 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:08.320 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:15:08.321 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:15:08.321 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:08.322 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:15:08.323 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:15:08.323 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:15:08.327 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:15:08.327 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:08.328 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:08.328 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:15:08.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:15:08.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:15:08.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:15:08.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_kill_daemon td/ceph-helpers 2026-02-12T19:15:08.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:266: test_kill_daemon: local dir=td/ceph-helpers 2026-02-12T19:15:08.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:267: test_kill_daemon: setup td/ceph-helpers 2026-02-12T19:15:08.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:15:08.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:15:08.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:15:08.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:15:08.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:15:08.331 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:08.331 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:08.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:08.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:08.331 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:08.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:08.332 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:15:08.333 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:15:08.333 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:15:08.334 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:15:08.334 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:15:08.335 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:15:08.336 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:08.336 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:15:08.336 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:15:08.336 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:08.337 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:15:08.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:15:08.338 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:15:08.340 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:15:08.340 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:08.340 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:08.340 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:15:08.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:15:08.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:15:08.341 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:15:08.343 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:15:08.343 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:08.343 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:08.343 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:15:08.344 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:15:08.346 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:15:08.346 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:15:08.346 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:15:08.346 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:268: test_kill_daemon: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:08.346 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:15:08.346 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:15:08.346 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:15:08.346 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:15:08.346 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:15:08.346 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:08.368 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:15:08.369 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:08.369 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:08.369 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:08.369 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:08.369 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:08.369 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:08.369 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:08.397 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:15:08.398 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:15:08.398 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:15:08.398 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:15:08.398 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:15:08.399 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:15:08.400 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:15:08.400 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:15:08.400 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:15:08.400 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:15:08.400 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:08.400 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:08.400 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:15:08.400 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:15:08.401 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:15:08.455 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:15:08.455 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:15:08.455 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:15:08.455 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:15:08.456 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:15:08.456 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:15:08.456 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:15:08.456 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:15:08.457 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:15:08.457 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:08.457 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:08.457 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:15:08.457 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:15:08.457 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:15:08.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:269: test_kill_daemon: run_mgr td/ceph-helpers x 2026-02-12T19:15:08.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:15:08.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:15:08.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:15:08.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:15:08.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:15:08.511 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:15:08.622 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:15:08.622 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:08.622 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:08.623 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:08.623 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:08.623 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:08.624 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:08.624 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:15:08.624 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:15:08.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:270: test_kill_daemon: run_osd td/ceph-helpers 0 2026-02-12T19:15:08.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:15:08.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:15:08.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:15:08.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:15:08.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:15:08.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:15:08.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:15:08.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:15:08.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:15:08.646 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:08.647 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:15:08.648 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:15:08.649 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=35e8fc43-4ec6-4a46-878c-1e644e5e6d26 2026-02-12T19:15:08.649 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 35e8fc43-4ec6-4a46-878c-1e644e5e6d26 2026-02-12T19:15:08.650 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 35e8fc43-4ec6-4a46-878c-1e644e5e6d26' 2026-02-12T19:15:08.650 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:15:08.658 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC8Jo5pv6n5JhAAL+UJhrMUVCrpPv44Bn7n7A== 2026-02-12T19:15:08.659 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC8Jo5pv6n5JhAAL+UJhrMUVCrpPv44Bn7n7A=="}' 2026-02-12T19:15:08.659 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 35e8fc43-4ec6-4a46-878c-1e644e5e6d26 -i td/ceph-helpers/0/new.json 2026-02-12T19:15:08.774 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:15:08.782 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:15:08.783 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC8Jo5pv6n5JhAAL+UJhrMUVCrpPv44Bn7n7A== --osd-uuid 35e8fc43-4ec6-4a46-878c-1e644e5e6d26 2026-02-12T19:15:08.799 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:08.791+0000 7f1f3f823a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:08.801 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:08.795+0000 7f1f3f823a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:08.802 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:08.795+0000 7f1f3f823a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:08.803 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:08.795+0000 7f1f3f823a40 -1 bdev(0x5631db5cc400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:15:08.803 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:08.795+0000 7f1f3f823a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:15:09.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:15:09.564 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:15:09.565 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:15:09.566 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:15:09.566 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:15:09.685 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:15:09.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:15:09.687 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:15:09.687 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:15:09.687 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:15:09.687 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:15:09.710 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:09.703+0000 7f0f90ce5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:09.713 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:09.707+0000 7f0f90ce5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:09.714 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:09.707+0000 7f0f90ce5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:09.794 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:15:09.794 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:15:09.794 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:15:09.795 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:15:09.795 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:15:09.795 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:09.795 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:15:09.795 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:15:09.796 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:09.796 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:09.900 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:15:10.162 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:10.155+0000 7f0f90ce5a40 -1 Falling back to public interface 2026-02-12T19:15:10.902 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:15:10.902 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:10.902 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:15:10.902 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:15:10.903 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:10.903 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:11.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:15:11.047 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:11.039+0000 7f0f90ce5a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:15:12.010 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:15:12.010 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:12.011 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:15:12.011 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:15:12.012 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:12.012 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:12.223 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/4156775523,v1:127.0.0.1:6801/4156775523] [v2:127.0.0.1:6802/4156775523,v1:127.0.0.1:6803/4156775523] exists,up 35e8fc43-4ec6-4a46-878c-1e644e5e6d26 2026-02-12T19:15:12.224 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:15:12.224 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:15:12.224 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:15:12.224 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:272: test_kill_daemon: name_prefix=osd 2026-02-12T19:15:12.225 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: find td/ceph-helpers 2026-02-12T19:15:12.225 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: grep 'osd[^/]*\.pid' 2026-02-12T19:15:12.226 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:273: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-12T19:15:12.226 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:280: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid 0 1 2026-02-12T19:15:12.226 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-12T19:15:12.227 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=93136 2026-02-12T19:15:12.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=0 2026-02-12T19:15:12.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local delays=1 2026-02-12T19:15:12.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-12T19:15:12.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-12T19:15:12.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 93136 2026-02-12T19:15:12.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-12T19:15:12.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-12T19:15:12.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 1 2026-02-12T19:15:13.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-12T19:15:13.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 93136 2026-02-12T19:15:13.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-12T19:15:13.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-12T19:15:13.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0 2026-02-12T19:15:13.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 1 2026-02-12T19:15:13.230 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:284: test_kill_daemon: kill_daemon td/ceph-helpers/osd.0.pid TERM 2026-02-12T19:15:13.230 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/osd.0.pid 2026-02-12T19:15:13.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=93136 2026-02-12T19:15:13.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-12T19:15:13.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-12T19:15:13.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-12T19:15:13.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-12T19:15:13.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 93136 2026-02-12T19:15:13.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-12T19:15:13.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-12T19:15:13.232 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-12T19:15:13.333 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-12T19:15:13.333 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 93136 2026-02-12T19:15:13.333 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-12T19:15:13.333 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-12T19:15:13.334 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.2 2026-02-12T19:15:13.534 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-12T19:15:13.534 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 93136 2026-02-12T19:15:13.535 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-12T19:15:13.535 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-12T19:15:13.535 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-12T19:15:13.535 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:287: test_kill_daemon: name_prefix=mgr 2026-02-12T19:15:13.535 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: find td/ceph-helpers 2026-02-12T19:15:13.536 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: grep 'mgr[^/]*\.pid' 2026-02-12T19:15:13.537 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:288: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-12T19:15:13.538 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:292: test_kill_daemon: kill_daemon td/ceph-helpers/mgr.x.pid TERM 2026-02-12T19:15:13.538 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mgr.x.pid 2026-02-12T19:15:13.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=92280 2026-02-12T19:15:13.539 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-12T19:15:13.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-12T19:15:13.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-12T19:15:13.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-12T19:15:13.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 92280 2026-02-12T19:15:13.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-12T19:15:13.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-12T19:15:13.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-12T19:15:13.641 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-12T19:15:13.641 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 92280 2026-02-12T19:15:13.641 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-12T19:15:13.641 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-12T19:15:13.641 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-12T19:15:13.641 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:295: test_kill_daemon: name_prefix=mon 2026-02-12T19:15:13.641 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: find td/ceph-helpers 2026-02-12T19:15:13.642 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: grep 'mon[^/]*\.pid' 2026-02-12T19:15:13.643 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:296: test_kill_daemon: for pidfile in $(find $dir 2>/dev/null | grep $name_prefix'[^/]*\.pid') 2026-02-12T19:15:13.643 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:300: test_kill_daemon: kill_daemon td/ceph-helpers/mon.a.pid TERM 2026-02-12T19:15:13.643 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: cat td/ceph-helpers/mon.a.pid 2026-02-12T19:15:13.644 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:246: kill_daemon: local pid=92195 2026-02-12T19:15:13.644 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:247: kill_daemon: local send_signal=TERM 2026-02-12T19:15:13.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:248: kill_daemon: local 'delays=0.1 0.2 1 1 1 2 3 5 5 5 10 10 20 60 60 60 120' 2026-02-12T19:15:13.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:249: kill_daemon: local exit_code=1 2026-02-12T19:15:13.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-12T19:15:13.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -TERM 92195 2026-02-12T19:15:13.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-12T19:15:13.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-12T19:15:13.645 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.1 2026-02-12T19:15:13.745 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-12T19:15:13.745 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 92195 2026-02-12T19:15:13.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:254: kill_daemon: exit_code=1 2026-02-12T19:15:13.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:259: kill_daemon: send_signal=0 2026-02-12T19:15:13.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:260: kill_daemon: sleep 0.2 2026-02-12T19:15:13.946 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:252: kill_daemon: for try in $delays 0 2026-02-12T19:15:13.947 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:253: kill_daemon: kill -0 92195 2026-02-12T19:15:13.947 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:256: kill_daemon: exit_code=0 2026-02-12T19:15:13.947 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:257: kill_daemon: break 2026-02-12T19:15:13.947 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:262: kill_daemon: return 0 2026-02-12T19:15:13.947 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:301: test_kill_daemon: timeout 5 ceph status 2026-02-12T19:15:18.951 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:304: test_kill_daemon: teardown td/ceph-helpers 2026-02-12T19:15:18.951 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:15:18.951 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:15:18.952 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:15:18.952 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:18.952 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:18.952 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:18.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:18.953 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:18.958 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:18.958 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:15:18.959 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:15:18.960 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:15:18.960 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:15:18.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:15:18.961 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:15:18.962 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:18.962 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:15:18.962 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:15:18.963 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:18.963 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:15:18.964 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:15:18.965 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:15:18.969 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:15:18.969 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:18.969 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:18.969 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:15:18.971 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:15:18.971 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:15:18.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:15:18.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_kill_daemons td/ceph-helpers 2026-02-12T19:15:18.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:357: test_kill_daemons: local dir=td/ceph-helpers 2026-02-12T19:15:18.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:358: test_kill_daemons: setup td/ceph-helpers 2026-02-12T19:15:18.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:15:18.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:15:18.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:15:18.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:15:18.972 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:15:18.972 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:18.973 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:18.973 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:18.973 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:18.973 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:18.973 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:18.974 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:15:18.974 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:15:18.975 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:15:18.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:15:18.976 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:15:18.976 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:15:18.977 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:18.977 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:15:18.978 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:15:18.978 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:18.979 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:15:18.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:15:18.980 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:15:18.981 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:15:18.982 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:18.982 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:18.982 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:15:18.983 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:15:18.983 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:15:18.983 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:15:18.984 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:15:18.985 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:18.985 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:18.985 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:15:18.986 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:15:18.987 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:15:18.987 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:15:18.987 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:15:18.987 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:359: test_kill_daemons: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:18.988 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:15:18.988 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:15:18.988 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:15:18.988 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:15:18.988 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:15:18.988 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:19.011 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:15:19.011 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:19.011 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:19.011 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:19.012 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:19.012 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:19.012 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:19.012 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:19.048 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:15:19.049 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:15:19.049 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:15:19.049 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:15:19.049 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:15:19.051 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:15:19.051 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:15:19.051 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:15:19.051 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:15:19.051 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:15:19.052 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:19.052 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:19.052 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:15:19.052 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:15:19.052 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:15:19.107 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:15:19.108 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:15:19.108 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:15:19.108 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:15:19.108 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:15:19.109 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:15:19.109 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:15:19.109 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:15:19.109 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:15:19.110 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:19.110 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:19.110 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:15:19.110 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:15:19.110 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:15:19.165 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:360: test_kill_daemons: run_mgr td/ceph-helpers x 2026-02-12T19:15:19.165 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:15:19.165 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:15:19.165 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:15:19.165 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:15:19.165 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:15:19.165 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:15:19.278 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:15:19.278 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:19.279 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:19.279 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:19.279 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:19.279 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:19.279 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:19.279 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:15:19.280 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:15:19.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:361: test_kill_daemons: run_osd td/ceph-helpers 0 2026-02-12T19:15:19.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:15:19.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:15:19.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:15:19.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:15:19.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:15:19.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:15:19.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:19.303 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:15:19.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:15:19.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:15:19.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:15:19.305 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:15:19.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d1c21938-ba45-4136-8ded-a682cec12c9a 2026-02-12T19:15:19.305 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d1c21938-ba45-4136-8ded-a682cec12c9a' 2026-02-12T19:15:19.306 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 d1c21938-ba45-4136-8ded-a682cec12c9a 2026-02-12T19:15:19.306 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:15:19.315 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDHJo5plM2AEhAAmiAyhGKAY7EehUvWEhLqmg== 2026-02-12T19:15:19.315 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDHJo5plM2AEhAAmiAyhGKAY7EehUvWEhLqmg=="}' 2026-02-12T19:15:19.315 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d1c21938-ba45-4136-8ded-a682cec12c9a -i td/ceph-helpers/0/new.json 2026-02-12T19:15:19.425 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:15:19.432 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:15:19.433 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDHJo5plM2AEhAAmiAyhGKAY7EehUvWEhLqmg== --osd-uuid d1c21938-ba45-4136-8ded-a682cec12c9a 2026-02-12T19:15:19.450 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:19.443+0000 7f8d56f4ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:19.451 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:19.443+0000 7f8d56f4ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:19.452 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:19.443+0000 7f8d56f4ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:19.453 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:19.443+0000 7f8d56f4ea40 -1 bdev(0x5597ad822400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:15:19.453 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:19.443+0000 7f8d56f4ea40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:15:20.156 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:15:20.156 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:15:20.157 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:15:20.157 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:15:20.158 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:15:20.269 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:15:20.270 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:15:20.271 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:15:20.271 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:15:20.271 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:15:20.271 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:15:20.285 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:20.279+0000 7f1e2182fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:20.287 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:20.279+0000 7f1e2182fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:20.288 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:20.279+0000 7f1e2182fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:20.386 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:15:20.386 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:15:20.386 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:15:20.386 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:15:20.386 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:15:20.386 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:20.386 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:15:20.387 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:15:20.387 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:20.387 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:20.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:15:20.730 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:20.723+0000 7f1e2182fa40 -1 Falling back to public interface 2026-02-12T19:15:21.401 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:21.395+0000 7f1e2182fa40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:15:21.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:15:21.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:21.496 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:15:21.496 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:15:21.497 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:21.500 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:21.631 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:15:22.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:15:22.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:22.634 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:15:22.634 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:15:22.634 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:22.634 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:22.853 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3815182946,v1:127.0.0.1:6801/3815182946] [v2:127.0.0.1:6802/3815182946,v1:127.0.0.1:6803/3815182946] exists,up d1c21938-ba45-4136-8ded-a682cec12c9a 2026-02-12T19:15:22.854 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:15:22.854 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:15:22.854 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:15:22.855 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:368: test_kill_daemons: kill_daemons td/ceph-helpers 0 osd 1 2026-02-12T19:15:22.855 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:22.855 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:22.855 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:22.855 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:22.855 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:23.862 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 1 2026-02-12T19:15:23.863 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:372: test_kill_daemons: kill_daemons td/ceph-helpers TERM osd 2026-02-12T19:15:23.863 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:23.864 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:23.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:23.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:23.864 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:24.172 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:24.172 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:376: test_kill_daemons: kill_daemons td/ceph-helpers TERM mgr 2026-02-12T19:15:24.173 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:24.173 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:24.173 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:24.173 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:24.174 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:24.281 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:24.281 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:380: test_kill_daemons: kill_daemons td/ceph-helpers TERM 2026-02-12T19:15:24.281 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:24.281 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:24.281 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:24.282 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:24.282 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:24.387 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:24.388 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:381: test_kill_daemons: timeout 5 ceph status 2026-02-12T19:15:29.393 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:382: test_kill_daemons: teardown td/ceph-helpers 2026-02-12T19:15:29.393 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:15:29.393 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:15:29.393 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:15:29.394 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:29.394 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:29.394 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:29.394 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:29.394 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:29.400 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:29.400 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:15:29.401 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:15:29.401 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:15:29.403 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:15:29.403 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:15:29.403 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:15:29.404 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:29.404 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:15:29.405 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:15:29.405 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:29.406 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:15:29.407 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:15:29.407 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:15:29.411 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:15:29.411 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:29.411 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:29.411 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:15:29.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:15:29.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:15:29.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:15:29.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_objectstore_tool td/ceph-helpers 2026-02-12T19:15:29.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1298: test_objectstore_tool: local dir=td/ceph-helpers 2026-02-12T19:15:29.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1300: test_objectstore_tool: setup td/ceph-helpers 2026-02-12T19:15:29.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:15:29.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:15:29.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:15:29.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:15:29.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:15:29.415 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:29.415 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:29.415 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:29.415 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:29.415 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:29.416 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:29.416 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:15:29.416 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:15:29.417 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:15:29.418 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:15:29.418 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:15:29.418 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:15:29.419 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:29.419 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:15:29.420 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:15:29.420 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:29.421 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:15:29.422 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:15:29.422 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:15:29.423 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:15:29.423 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:29.424 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:29.424 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:15:29.425 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:15:29.425 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:15:29.425 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:15:29.426 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:15:29.426 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:29.427 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:29.427 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:15:29.428 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:15:29.429 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:15:29.429 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:15:29.429 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:15:29.429 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1301: test_objectstore_tool: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:29.430 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:15:29.430 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:15:29.430 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:15:29.430 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:15:29.430 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:15:29.430 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:29.450 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:15:29.451 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:29.451 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:29.452 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:29.452 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:29.452 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:29.452 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:29.452 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:29.479 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:15:29.480 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:15:29.480 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:15:29.481 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:15:29.481 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:15:29.482 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:15:29.482 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:15:29.482 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:15:29.482 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:15:29.482 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:15:29.482 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:29.482 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:29.482 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:15:29.483 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:15:29.483 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:15:29.538 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:15:29.538 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:15:29.538 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:15:29.538 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:15:29.538 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:15:29.539 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:15:29.539 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:15:29.539 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:15:29.539 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:15:29.539 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:29.540 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:29.540 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:15:29.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:15:29.540 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:15:29.593 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1302: test_objectstore_tool: local osd=0 2026-02-12T19:15:29.594 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1303: test_objectstore_tool: run_mgr td/ceph-helpers x 2026-02-12T19:15:29.594 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:15:29.594 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:15:29.594 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:15:29.594 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:15:29.594 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:15:29.594 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:15:29.705 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:15:29.705 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:29.706 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:29.706 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:29.706 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:29.706 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:29.706 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:29.706 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:15:29.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:15:29.728 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1304: test_objectstore_tool: run_osd td/ceph-helpers 0 2026-02-12T19:15:29.728 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:15:29.728 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:15:29.728 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:15:29.729 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:15:29.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:15:29.731 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:15:29.731 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:15:29.731 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:15:29.731 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:15:29.731 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:15:29.731 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:15:29.732 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:15:29.732 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2adb6e17-c024-43e3-921a-c56f7619f4da 2026-02-12T19:15:29.732 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2adb6e17-c024-43e3-921a-c56f7619f4da' 2026-02-12T19:15:29.732 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 2adb6e17-c024-43e3-921a-c56f7619f4da 2026-02-12T19:15:29.732 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:15:29.742 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDRJo5pe2/1KxAAWTCz2huactc4Ed/BAh1wBg== 2026-02-12T19:15:29.742 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDRJo5pe2/1KxAAWTCz2huactc4Ed/BAh1wBg=="}' 2026-02-12T19:15:29.742 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2adb6e17-c024-43e3-921a-c56f7619f4da -i td/ceph-helpers/0/new.json 2026-02-12T19:15:29.858 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:15:29.865 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:15:29.867 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDRJo5pe2/1KxAAWTCz2huactc4Ed/BAh1wBg== --osd-uuid 2adb6e17-c024-43e3-921a-c56f7619f4da 2026-02-12T19:15:29.883 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:29.875+0000 7f642e2dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:29.885 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:29.879+0000 7f642e2dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:29.886 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:29.879+0000 7f642e2dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:29.886 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:29.879+0000 7f642e2dca40 -1 bdev(0x563318580400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:15:29.886 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:29.879+0000 7f642e2dca40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:15:30.669 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:15:30.669 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:15:30.670 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:15:30.670 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:15:30.670 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:15:30.789 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:15:30.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:15:30.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:15:30.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:15:30.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:15:30.790 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:15:30.814 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:30.807+0000 7fc90b6a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:30.817 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:30.811+0000 7fc90b6a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:30.818 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:30.811+0000 7fc90b6a6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:30.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:15:30.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:15:30.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:15:30.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:15:30.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:15:30.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:30.898 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:15:30.899 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:15:30.899 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:30.899 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:31.005 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:15:31.207 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:31.199+0000 7fc90b6a6a40 -1 Falling back to public interface 2026-02-12T19:15:32.007 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:15:32.007 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:32.008 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:15:32.008 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:15:32.009 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:32.009 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:32.118 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:15:32.119 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:32.111+0000 7fc90b6a6a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:15:33.120 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:15:33.120 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:33.120 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:15:33.121 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:15:33.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:33.121 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:33.331 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1054101206,v1:127.0.0.1:6801/1054101206] [v2:127.0.0.1:6802/1054101206,v1:127.0.0.1:6803/1054101206] exists,up 2adb6e17-c024-43e3-921a-c56f7619f4da 2026-02-12T19:15:33.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:15:33.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:15:33.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:15:33.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1305: test_objectstore_tool: create_rbd_pool 2026-02-12T19:15:33.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:15:33.531 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:15:33.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:15:33.540 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:15:33.785 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:15:33.796 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:15:34.797 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:15:35.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1306: test_objectstore_tool: wait_for_clean 2026-02-12T19:15:35.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:15:35.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:15:35.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:15:35.091 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:15:35.092 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:15:35.092 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:15:35.092 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:15:35.092 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:15:35.093 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:15:35.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:15:35.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:15:35.198 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:15:35.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:15:35.199 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:15:35.199 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:15:35.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:15:35.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:15:35.410 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:15:35.410 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:15:35.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-12T19:15:35.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-12T19:15:35.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-12T19:15:35.494 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:15:35.494 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-12T19:15:35.494 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:15:35.495 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:15:35.495 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-12T19:15:35.496 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:15:35.497 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-12T19:15:35.497 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-12T19:15:35.498 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836483 2026-02-12T19:15:35.498 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:35.712 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-12T19:15:35.712 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:15:36.713 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:15:36.714 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:36.925 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-12T19:15:36.925 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:15:36.926 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:15:36.926 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:15:37.209 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:15:37.209 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:15:37.209 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:15:37.210 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:15:37.210 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:15:37.210 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:15:37.210 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:15:37.210 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:15:37.412 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:15:37.412 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:15:37.413 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:15:37.413 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:15:37.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:15:37.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:15:37.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:15:37.686 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1307: test_objectstore_tool: rados --pool rbd put GROUP /etc/group 2026-02-12T19:15:37.706 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1308: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 GROUP get-bytes 2026-02-12T19:15:37.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-12T19:15:37.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T19:15:37.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-12T19:15:37.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T19:15:37.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 GROUP get-bytes 2026-02-12T19:15:37.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-12T19:15:37.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T19:15:37.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-12T19:15:37.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T19:15:37.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-12T19:15:37.707 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1309: test_objectstore_tool: diff - /etc/group 2026-02-12T19:15:37.708 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:37.708 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:37.708 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:37.708 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:37.708 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:38.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:38.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 GROUP get-bytes 2026-02-12T19:15:38.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-12T19:15:38.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T19:15:38.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-12T19:15:38.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T19:15:38.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-12T19:15:38.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 GROUP get-bytes 2026-02-12T19:15:38.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/ceph-helpers 0 2026-02-12T19:15:38.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ceph-helpers 2026-02-12T19:15:38.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-12T19:15:38.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-12T19:15:38.203 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-12T19:15:38.204 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:15:38.205 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:15:38.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-12T19:15:38.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-12T19:15:38.206 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:15:38.207 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-12T19:15:38.207 INFO:tasks.workunit.client.0.trial010.stderr:start osd.0 2026-02-12T19:15:38.207 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:15:38.207 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ceph-helpers/0/whoami 2026-02-12T19:15:38.207 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-12T19:15:38.207 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-12T19:15:38.207 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-12T19:15:38.207 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-12T19:15:38.222 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:38.215+0000 7fc095c36a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:38.223 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:38.215+0000 7fc095c36a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:38.224 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:38.215+0000 7fc095c36a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:38.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-12T19:15:38.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:15:38.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:15:38.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:15:38.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:15:38.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:38.414 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:15:38.414 INFO:tasks.workunit.client.0.trial010.stderr:0 2026-02-12T19:15:38.415 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:38.415 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:38.578 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:38.571+0000 7fc095c36a40 -1 Falling back to public interface 2026-02-12T19:15:38.631 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:15:39.256 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:39.247+0000 7fc095c36a40 -1 osd.0 12 log_to_monitors true 2026-02-12T19:15:39.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:15:39.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:39.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:15:39.632 INFO:tasks.workunit.client.0.trial010.stderr:1 2026-02-12T19:15:39.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:39.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:39.848 INFO:tasks.workunit.client.0.trial010.stderr:osd.0 up in weight 1 up_from 15 up_thru 15 down_at 13 last_clean_interval [5,12) [v2:127.0.0.1:6800/3596625377,v1:127.0.0.1:6801/3596625377] [v2:127.0.0.1:6802/3596625377,v1:127.0.0.1:6803/3596625377] exists,up 2adb6e17-c024-43e3-921a-c56f7619f4da 2026-02-12T19:15:39.849 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:15:39.849 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:15:39.849 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:15:39.849 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-12T19:15:39.849 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:15:39.849 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:15:39.849 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:15:39.849 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:15:39.850 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:15:39.850 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:15:39.850 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:15:39.850 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:15:39.850 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:15:39.950 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:15:39.950 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:15:39.950 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:15:39.950 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:15:39.951 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:15:39.951 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:15:40.166 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:15:40.166 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:15:40.166 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:15:40.166 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:15:40.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509443 2026-02-12T19:15:40.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509443 2026-02-12T19:15:40.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-64424509443' 2026-02-12T19:15:40.246 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:15:40.247 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-64424509443 2026-02-12T19:15:40.247 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:15:40.247 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:15:40.248 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-64424509443 2026-02-12T19:15:40.248 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:15:40.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=64424509443 2026-02-12T19:15:40.250 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 64424509443' 2026-02-12T19:15:40.250 INFO:tasks.workunit.client.0.trial010.stderr:waiting osd.0 seq 64424509443 2026-02-12T19:15:40.250 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:40.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 64424509443 2026-02-12T19:15:40.467 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:15:41.468 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:15:41.469 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:41.684 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 1 -lt 64424509443 2026-02-12T19:15:41.684 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:15:42.685 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T19:15:42.686 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:42.907 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-12T19:15:42.907 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:15:42.908 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:15:42.908 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:15:43.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:15:43.200 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:15:43.200 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:15:43.200 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:15:43.200 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:15:43.200 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:15:43.201 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:15:43.201 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:15:43.409 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:15:43.410 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:15:43.410 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:15:43.410 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:15:43.696 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:15:43.696 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:15:43.696 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:15:43.696 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1310: test_objectstore_tool: objectstore_tool td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-12T19:15:43.696 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/ceph-helpers 2026-02-12T19:15:43.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-12T19:15:43.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=0 2026-02-12T19:15:43.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-12T19:15:43.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-12T19:15:43.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/ceph-helpers 2026-02-12T19:15:43.697 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-12T19:15:43.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=0 2026-02-12T19:15:43.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-12T19:15:43.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/ceph-helpers TERM osd.0 2026-02-12T19:15:43.698 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:43.698 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:43.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:43.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:43.698 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:44.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:44.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/ceph-helpers 0 NOTEXISTS get-bytes 2026-02-12T19:15:44.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/ceph-helpers 2026-02-12T19:15:44.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-12T19:15:44.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-12T19:15:44.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-12T19:15:44.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/ceph-helpers/0 2026-02-12T19:15:44.003 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/ceph-helpers/0 NOTEXISTS get-bytes 2026-02-12T19:15:44.075 INFO:tasks.workunit.client.0.trial010.stderr:No object id 'NOTEXISTS' found or invalid JSON specified 2026-02-12T19:15:44.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-12T19:15:44.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: return 1 2026-02-12T19:15:44.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: return 1 2026-02-12T19:15:44.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1311: test_objectstore_tool: teardown td/ceph-helpers 2026-02-12T19:15:44.186 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:15:44.187 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:15:44.187 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:15:44.187 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:44.187 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:44.187 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:44.187 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:44.187 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:44.294 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:44.294 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:15:44.295 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:15:44.295 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:15:44.297 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:15:44.297 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:15:44.297 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:15:44.298 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:44.298 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:15:44.299 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:15:44.299 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:44.300 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:15:44.301 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:15:44.301 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:15:44.307 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:15:44.307 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:44.307 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:44.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:15:44.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:15:44.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_pg_schedule_scrub td/ceph-helpers 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1987: test_pg_schedule_scrub: local dir=td/ceph-helpers 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1989: test_pg_schedule_scrub: setup td/ceph-helpers 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:15:44.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:15:44.311 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:15:44.311 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:15:44.311 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:15:44.312 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:15:44.313 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:15:44.313 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:15:44.314 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:15:44.314 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:15:44.314 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:15:44.315 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:44.315 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:15:44.316 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:15:44.316 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:15:44.317 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:15:44.318 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:15:44.318 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:15:44.320 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:15:44.320 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:44.320 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:44.320 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:15:44.321 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:15:44.321 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:15:44.321 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:15:44.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:15:44.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:44.323 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:44.323 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:15:44.324 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:15:44.325 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:15:44.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:15:44.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:15:44.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1990: test_pg_schedule_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:44.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:15:44.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:15:44.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:15:44.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:15:44.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:15:44.326 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:44.349 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:15:44.349 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:44.349 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:44.349 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:44.349 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:44.349 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:44.350 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:44.350 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:15:44.379 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:15:44.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:15:44.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:15:44.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:15:44.380 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:15:44.381 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:15:44.382 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:15:44.382 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:15:44.382 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:15:44.382 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:15:44.382 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:44.382 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:44.382 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:15:44.383 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:15:44.383 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:15:44.437 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:15:44.438 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:15:44.438 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:15:44.438 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:15:44.438 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:15:44.438 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:15:44.438 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:15:44.438 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:15:44.439 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:15:44.439 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:44.439 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:44.439 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:15:44.439 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:15:44.440 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:15:44.492 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1991: test_pg_schedule_scrub: run_mgr td/ceph-helpers x --mgr_stats_period=1 2026-02-12T19:15:44.492 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:15:44.492 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:15:44.492 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:15:44.493 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:15:44.493 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:15:44.493 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:15:44.602 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:15:44.602 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:44.602 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:44.602 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:44.603 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:44.603 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:44.603 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:44.603 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:15:44.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr --mgr_stats_period=1 2026-02-12T19:15:44.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1992: test_pg_schedule_scrub: run_osd td/ceph-helpers 0 2026-02-12T19:15:44.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:15:44.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:15:44.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:15:44.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:15:44.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:15:44.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:15:44.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:15:44.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:15:44.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:15:44.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:15:44.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:15:44.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:15:44.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:15:44.628 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:15:44.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:15:44.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:15:44.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:15:44.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:15:44.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:15:44.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:15:44.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:15:44.629 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:15:44.630 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:15:44.630 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=942628cb-de09-4914-9083-ce5e5ec8020e 2026-02-12T19:15:44.630 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 942628cb-de09-4914-9083-ce5e5ec8020e' 2026-02-12T19:15:44.630 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 942628cb-de09-4914-9083-ce5e5ec8020e 2026-02-12T19:15:44.631 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:15:44.640 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgJo5puYXZJRAAUEt+it/4+Mfh8xut14WjfA== 2026-02-12T19:15:44.640 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgJo5puYXZJRAAUEt+it/4+Mfh8xut14WjfA=="}' 2026-02-12T19:15:44.640 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 942628cb-de09-4914-9083-ce5e5ec8020e -i td/ceph-helpers/0/new.json 2026-02-12T19:15:44.755 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:15:44.764 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:15:44.765 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgJo5puYXZJRAAUEt+it/4+Mfh8xut14WjfA== --osd-uuid 942628cb-de09-4914-9083-ce5e5ec8020e 2026-02-12T19:15:44.781 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:44.770+0000 7f7509278a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:44.782 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:44.774+0000 7f7509278a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:44.783 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:44.774+0000 7f7509278a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:44.784 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:44.774+0000 7f7509278a40 -1 bdev(0x559814328400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:15:44.784 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:44.774+0000 7f7509278a40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:15:45.505 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:15:45.505 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:15:45.505 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:15:45.506 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:15:45.506 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:15:45.625 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:15:45.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:15:45.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:15:45.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:15:45.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:15:45.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:15:45.649 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:45.642+0000 7f105f666a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:45.652 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:45.642+0000 7f105f666a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:45.653 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:45.646+0000 7f105f666a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:15:45.734 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:15:45.734 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:15:45.734 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:15:45.734 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:15:45.734 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:15:45.734 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:45.735 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:15:45.735 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:15:45.735 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:45.735 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:45.850 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:15:46.070 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:46.062+0000 7f105f666a40 -1 Falling back to public interface 2026-02-12T19:15:46.782 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:15:46.774+0000 7f105f666a40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:15:46.852 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:15:46.852 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:46.852 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:15:46.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:15:46.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:46.853 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:46.964 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:15:47.965 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:15:47.966 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:15:47.966 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:15:47.967 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:15:47.967 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:15:47.967 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:15:48.177 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1453881543,v1:127.0.0.1:6801/1453881543] [v2:127.0.0.1:6802/1453881543,v1:127.0.0.1:6803/1453881543] exists,up 942628cb-de09-4914-9083-ce5e5ec8020e 2026-02-12T19:15:48.178 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:15:48.178 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:15:48.178 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:15:48.178 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1993: test_pg_schedule_scrub: create_rbd_pool 2026-02-12T19:15:48.179 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:15:48.376 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:15:48.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:15:48.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:15:48.622 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:15:48.631 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:15:49.632 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:15:49.911 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1994: test_pg_schedule_scrub: wait_for_clean 2026-02-12T19:15:49.911 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:15:49.911 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:15:49.911 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:15:49.912 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:15:49.912 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:15:49.912 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:15:49.912 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:15:49.913 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:15:49.913 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:15:50.014 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:15:50.014 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:15:50.014 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:15:50.014 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:15:50.015 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:15:50.015 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:15:50.228 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:15:50.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:15:50.229 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:15:50.229 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:15:50.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-12T19:15:50.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-12T19:15:50.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485' 2026-02-12T19:15:50.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:15:50.308 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836485 2026-02-12T19:15:50.308 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:15:50.309 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:15:50.309 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-12T19:15:50.309 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:15:50.310 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836485 2026-02-12T19:15:50.311 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-12T19:15:50.311 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836485 2026-02-12T19:15:50.311 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:50.521 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836485 2026-02-12T19:15:50.521 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:15:51.522 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:15:51.523 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:51.742 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-12T19:15:51.743 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:15:51.743 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:15:51.743 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:15:52.026 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:15:52.026 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:15:52.026 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:15:52.027 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:15:52.027 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:15:52.027 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:15:52.027 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:15:52.027 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:15:52.235 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:15:52.235 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:15:52.236 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:15:52.236 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:15:52.512 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:15:52.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:15:52.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:15:52.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1995: test_pg_schedule_scrub: pg_schedule_scrub 1.0 2026-02-12T19:15:52.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1969: pg_schedule_scrub: local pgid=1.0 2026-02-12T19:15:52.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1971: pg_schedule_scrub: wait_for_pg_clean 1.0 2026-02-12T19:15:52.513 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-12T19:15:52.513 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-12T19:15:52.513 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:15:52.513 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:15:52.514 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:15:52.514 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:15:52.514 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:15:52.783 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-12T19:15:52.783 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-12T19:15:52.783 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-12T19:15:52.783 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-12T19:15:52.783 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:15:52.783 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:15:53.010 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:15:53.011 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:15:53.011 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:15:53.011 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:15:53.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-12T19:15:53.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-12T19:15:53.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836490' 2026-02-12T19:15:53.090 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:15:53.091 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836490 2026-02-12T19:15:53.091 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:15:53.092 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:15:53.093 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-12T19:15:53.093 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:15:53.094 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836490 2026-02-12T19:15:53.094 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-12T19:15:53.095 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836490 2026-02-12T19:15:53.095 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:53.319 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836490 2026-02-12T19:15:53.320 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:15:54.320 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:15:54.321 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:54.541 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836487 -lt 21474836490 2026-02-12T19:15:54.541 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:15:55.543 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T19:15:55.543 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:15:55.752 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836491 -lt 21474836490 2026-02-12T19:15:55.752 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-12T19:15:55.753 INFO:tasks.workunit.client.0.trial010.stdout:#---------- 1.0 loop 0 2026-02-12T19:15:55.753 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-12T19:15:55.754 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-12T19:15:55.754 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-12T19:15:55.754 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-12T19:15:55.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-12T19:15:55.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-12T19:15:55.833 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-12T19:15:55.833 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-12T19:15:55.833 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-12T19:15:55.833 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-12T19:15:55.833 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: get_last_scrub_stamp 1.0 2026-02-12T19:15:55.833 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-12T19:15:55.833 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-12T19:15:55.833 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-12T19:15:55.834 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-12T19:15:56.041 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1972: pg_schedule_scrub: local last_scrub=2026-02-12T19:15:48.565029+0000 2026-02-12T19:15:56.041 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1973: pg_schedule_scrub: ceph tell 1.0 schedule-scrub 2026-02-12T19:15:56.116 INFO:tasks.workunit.client.0.trial010.stdout:{ 2026-02-12T19:15:56.116 INFO:tasks.workunit.client.0.trial010.stdout: "deep": false, 2026-02-12T19:15:56.116 INFO:tasks.workunit.client.0.trial010.stdout: "must": false, 2026-02-12T19:15:56.116 INFO:tasks.workunit.client.0.trial010.stdout: "stamp": "2026-02-10T07:15:56.112742+0000" 2026-02-12T19:15:56.116 INFO:tasks.workunit.client.0.trial010.stdout:} 2026-02-12T19:15:56.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1974: pg_schedule_scrub: wait_for_scrub 1.0 2026-02-12T19:15:48.565029+0000 2026-02-12T19:15:56.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-12T19:15:56.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-12T19:15:48.565029+0000 2026-02-12T19:15:56.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-12T19:15:56.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-12T19:15:56.124 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-12T19:15:56.124 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-12T19:15:56.124 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-12T19:15:56.124 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-12T19:15:56.125 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-12T19:15:56.125 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-12T19:15:56.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-12T19:15:48.565029+0000 '>' 2026-02-12T19:15:48.565029+0000 2026-02-12T19:15:56.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-12T19:15:57.334 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-12T19:15:57.334 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-12T19:15:57.335 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-12T19:15:57.335 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-12T19:15:57.335 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-12T19:15:57.335 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-12T19:15:57.335 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-12T19:15:57.545 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-12T19:15:48.565029+0000 '>' 2026-02-12T19:15:48.565029+0000 2026-02-12T19:15:57.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-12T19:15:58.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-12T19:15:58.546 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-12T19:15:58.547 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-12T19:15:58.547 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-12T19:15:58.547 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-12T19:15:58.547 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-12T19:15:58.547 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-12T19:15:58.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-12T19:15:48.565029+0000 '>' 2026-02-12T19:15:48.565029+0000 2026-02-12T19:15:58.751 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-12T19:15:59.752 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-12T19:15:59.753 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-12T19:15:59.753 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-12T19:15:59.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-12T19:15:59.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-12T19:15:59.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-12T19:15:59.754 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-12T19:15:59.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-10T07:15:56.112742+0000 '>' 2026-02-12T19:15:48.565029+0000 2026-02-12T19:15:59.961 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-12T19:16:00.962 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-12T19:16:00.962 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-12T19:16:00.963 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-12T19:16:00.963 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-12T19:16:00.963 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-12T19:16:00.963 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-12T19:16:00.963 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-12T19:16:01.167 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-12T19:15:59.851982+0000 '>' 2026-02-12T19:15:48.565029+0000 2026-02-12T19:16:01.167 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-12T19:16:01.168 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1996: test_pg_schedule_scrub: teardown td/ceph-helpers 2026-02-12T19:16:01.168 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:16:01.168 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:16:01.168 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:16:01.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:16:01.168 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:16:01.168 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:16:01.168 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:16:01.168 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:16:01.276 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:16:01.277 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:16:01.277 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:16:01.277 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:16:01.278 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:16:01.278 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:16:01.279 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:16:01.280 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:16:01.280 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:16:01.280 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:16:01.281 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:16:01.282 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:16:01.282 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:16:01.283 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:16:01.288 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:16:01.289 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:16:01.289 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:16:01.289 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:16:01.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:16:01.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:16:01.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2425: run_tests: for func in $funcs 2026-02-12T19:16:01.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2426: run_tests: test_pg_scrub td/ceph-helpers 2026-02-12T19:16:01.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1941: test_pg_scrub: local dir=td/ceph-helpers 2026-02-12T19:16:01.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1943: test_pg_scrub: setup td/ceph-helpers 2026-02-12T19:16:01.291 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ceph-helpers 2026-02-12T19:16:01.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ceph-helpers 2026-02-12T19:16:01.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T19:16:01.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-12T19:16:01.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T19:16:01.292 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:16:01.292 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:16:01.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:16:01.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:16:01.292 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:16:01.293 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:16:01.293 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T19:16:01.294 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T19:16:01.295 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T19:16:01.295 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T19:16:01.296 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T19:16:01.296 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T19:16:01.297 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:16:01.297 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T19:16:01.298 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T19:16:01.298 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T19:16:01.299 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T19:16:01.300 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-12T19:16:01.300 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T19:16:01.301 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T19:16:01.301 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:16:01.301 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:16:01.301 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T19:16:01.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T19:16:01.302 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T19:16:01.303 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ceph-helpers 2026-02-12T19:16:01.304 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-12T19:16:01.304 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:16:01.304 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:16:01.304 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.52248 2026-02-12T19:16:01.306 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-12T19:16:01.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-12T19:16:01.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-12T19:16:01.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ceph-helpers 1' TERM HUP INT 2026-02-12T19:16:01.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1944: test_pg_scrub: run_mon td/ceph-helpers a --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:16:01.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ceph-helpers 2026-02-12T19:16:01.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-12T19:16:01.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-12T19:16:01.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-12T19:16:01.307 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ceph-helpers/a 2026-02-12T19:16:01.308 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ceph-helpers/a --run-dir=td/ceph-helpers --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:16:01.328 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-12T19:16:01.329 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:16:01.329 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:16:01.329 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:16:01.330 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:16:01.330 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:16:01.330 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:16:01.330 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ceph-helpers/a '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --mon-cluster-log-file=td/ceph-helpers/log --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_pool_default_size=1 --mon_allow_pool_size_one=true 2026-02-12T19:16:01.358 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-12T19:16:01.359 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-12T19:16:01.359 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:16:01.359 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:16:01.359 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-12T19:16:01.360 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-12T19:16:01.360 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:16:01.360 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:16:01.360 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:16:01.360 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:16:01.361 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:16:01.361 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:16:01.361 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:16:01.361 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:16:01.361 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get fsid 2026-02-12T19:16:01.416 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-12T19:16:01.416 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-12T19:16:01.416 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-12T19:16:01.417 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-12T19:16:01.417 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-12T19:16:01.418 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-12T19:16:01.418 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-12T19:16:01.418 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-12T19:16:01.418 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-12T19:16:01.418 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:16:01.418 INFO:tasks.workunit.client.0.trial010.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:16:01.418 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.52248/ceph-mon.a.asok 2026-02-12T19:16:01.418 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-12T19:16:01.418 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.52248/ceph-mon.a.asok config get mon_host 2026-02-12T19:16:01.474 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1945: test_pg_scrub: run_mgr td/ceph-helpers x 2026-02-12T19:16:01.474 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ceph-helpers 2026-02-12T19:16:01.474 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-12T19:16:01.474 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-12T19:16:01.474 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-12T19:16:01.474 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ceph-helpers/x 2026-02-12T19:16:01.474 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-12T19:16:01.589 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-12T19:16:01.589 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:16:01.589 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:16:01.590 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:16:01.590 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:16:01.590 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:16:01.590 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:16:01.591 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:16:01.591 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ceph-helpers/x '--log-file=td/ceph-helpers/$name.log' '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --run-dir=td/ceph-helpers '--pid-file=td/ceph-helpers/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-12T19:16:01.612 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1946: test_pg_scrub: run_osd td/ceph-helpers 0 2026-02-12T19:16:01.612 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ceph-helpers 2026-02-12T19:16:01.612 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-12T19:16:01.612 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-12T19:16:01.612 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ceph-helpers/0 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 ' 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ceph-helpers/0' 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ceph-helpers/0/journal' 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ceph-helpers' 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-12T19:16:01.613 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ceph-helpers/$name.log' 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ceph-helpers/$name.pid' 2026-02-12T19:16:01.614 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-12T19:16:01.615 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-12T19:16:01.615 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-12T19:16:01.615 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-12T19:16:01.615 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-12T19:16:01.615 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-12T19:16:01.615 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ceph-helpers/0 2026-02-12T19:16:01.616 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-12T19:16:01.616 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=10608a2a-f006-430f-928e-a3d593dfdea9 2026-02-12T19:16:01.616 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 10608a2a-f006-430f-928e-a3d593dfdea9' 2026-02-12T19:16:01.616 INFO:tasks.workunit.client.0.trial010.stdout:add osd0 10608a2a-f006-430f-928e-a3d593dfdea9 2026-02-12T19:16:01.616 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-12T19:16:01.626 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDxJo5pP1YNJRAApz/5UMzw1TiUYvMPu4dFhg== 2026-02-12T19:16:01.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDxJo5pP1YNJRAApz/5UMzw1TiUYvMPu4dFhg=="}' 2026-02-12T19:16:01.627 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 10608a2a-f006-430f-928e-a3d593dfdea9 -i td/ceph-helpers/0/new.json 2026-02-12T19:16:01.739 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:16:01.747 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ceph-helpers/0/new.json 2026-02-12T19:16:01.748 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDxJo5pP1YNJRAApz/5UMzw1TiUYvMPu4dFhg== --osd-uuid 10608a2a-f006-430f-928e-a3d593dfdea9 2026-02-12T19:16:01.764 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:16:01.754+0000 7f4c0e43ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:16:01.766 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:16:01.758+0000 7f4c0e43ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:16:01.767 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:16:01.758+0000 7f4c0e43ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:16:01.768 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:16:01.758+0000 7f4c0e43ca40 -1 bdev(0x55a6ff86e400 td/ceph-helpers/0/block) open stat got: (1) Operation not permitted 2026-02-12T19:16:01.768 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:16:01.758+0000 7f4c0e43ca40 -1 bluestore(td/ceph-helpers/0) _read_fsid unparsable uuid 2026-02-12T19:16:02.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ceph-helpers/0/keyring 2026-02-12T19:16:02.508 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-12T19:16:02.509 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-12T19:16:02.510 INFO:tasks.workunit.client.0.trial010.stdout:adding osd0 key to auth repository 2026-02-12T19:16:02.510 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ceph-helpers/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-12T19:16:02.632 INFO:tasks.workunit.client.0.trial010.stdout:start osd.0 2026-02-12T19:16:02.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-12T19:16:02.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --cluster ceph --fsid=87fd2eab-282e-43bf-b5a2-5c7fac36961f --auth-supported=none --mon-host=127.0.0.1:7109 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ceph-helpers/0 --osd-journal=td/ceph-helpers/0/journal --chdir= --run-dir=td/ceph-helpers '--admin-socket=/tmp/ceph-asok.52248/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ceph-helpers/$name.log' '--pid-file=td/ceph-helpers/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-12T19:16:02.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-12T19:16:02.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-12T19:16:02.633 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-12T19:16:02.659 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:16:02.650+0000 7fe1f84dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:16:02.661 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:16:02.654+0000 7fe1f84dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:16:02.662 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:16:02.654+0000 7fe1f84dca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-12T19:16:02.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-12T19:16:02.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-12T19:16:02.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-12T19:16:02.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-12T19:16:02.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-12T19:16:02.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:16:02.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-12T19:16:02.746 INFO:tasks.workunit.client.0.trial010.stdout:0 2026-02-12T19:16:02.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:16:02.746 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:16:02.886 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:16:03.114 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:16:03.106+0000 7fe1f84dca40 -1 Falling back to public interface 2026-02-12T19:16:03.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:16:03.888 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:16:03.888 INFO:tasks.workunit.client.0.trial010.stdout:1 2026-02-12T19:16:03.889 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-12T19:16:03.889 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:16:03.889 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:16:03.997 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-12T19:16:04.051 INFO:tasks.workunit.client.0.trial010.stderr:2026-02-12T19:16:04.042+0000 7fe1f84dca40 -1 osd.0 0 log_to_monitors true 2026-02-12T19:16:04.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-12T19:16:04.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-12T19:16:04.999 INFO:tasks.workunit.client.0.trial010.stdout:2 2026-02-12T19:16:04.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-12T19:16:04.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-12T19:16:04.999 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-12T19:16:05.218 INFO:tasks.workunit.client.0.trial010.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/65075307,v1:127.0.0.1:6801/65075307] [v2:127.0.0.1:6802/65075307,v1:127.0.0.1:6803/65075307] exists,up 10608a2a-f006-430f-928e-a3d593dfdea9 2026-02-12T19:16:05.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-12T19:16:05.219 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-12T19:16:05.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-12T19:16:05.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1947: test_pg_scrub: create_rbd_pool 2026-02-12T19:16:05.220 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-12T19:16:05.433 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' does not exist 2026-02-12T19:16:05.443 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-12T19:16:05.443 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-12T19:16:05.718 INFO:tasks.workunit.client.0.trial010.stderr:pool 'rbd' already exists 2026-02-12T19:16:05.728 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-12T19:16:06.730 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-12T19:16:06.973 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1948: test_pg_scrub: wait_for_clean 2026-02-12T19:16:06.974 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-12T19:16:06.974 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-12T19:16:06.974 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-12T19:16:06.974 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-12T19:16:06.975 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:16:06.975 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:16:06.975 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:16:06.975 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:16:06.975 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:16:07.077 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-12T19:16:07.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-12T19:16:07.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-12T19:16:07.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-12T19:16:07.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:16:07.078 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:16:07.296 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:16:07.296 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:16:07.296 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:16:07.296 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:16:07.374 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836483 2026-02-12T19:16:07.374 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836483 2026-02-12T19:16:07.374 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836483' 2026-02-12T19:16:07.374 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:16:07.375 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836483 2026-02-12T19:16:07.375 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:16:07.375 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:16:07.376 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836483 2026-02-12T19:16:07.376 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:16:07.378 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836483 2026-02-12T19:16:07.378 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836483' 2026-02-12T19:16:07.378 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836483 2026-02-12T19:16:07.378 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:16:07.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836481 -lt 21474836483 2026-02-12T19:16:07.589 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:16:08.590 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:16:08.590 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:16:08.814 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836483 2026-02-12T19:16:08.814 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-12T19:16:08.815 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:16:08.815 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:16:09.101 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-12T19:16:09.101 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-12T19:16:09.101 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-12T19:16:09.101 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-12T19:16:09.101 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-12T19:16:09.101 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-12T19:16:09.102 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-12T19:16:09.102 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-12T19:16:09.317 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-12T19:16:09.318 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-12T19:16:09.318 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-12T19:16:09.318 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-12T19:16:09.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-12T19:16:09.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-12T19:16:09.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-12T19:16:09.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1949: test_pg_scrub: pg_scrub 1.0 2026-02-12T19:16:09.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-12T19:16:09.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-12T19:16:09.604 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-12T19:16:09.604 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 90 1 3 2026-02-12T19:16:09.604 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:16:09.604 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:16:09.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:16:09.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:16:09.605 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:16:09.865 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3' '3') 2026-02-12T19:16:09.865 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-12T19:16:09.866 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-12T19:16:09.866 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-12T19:16:09.866 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-12T19:16:09.866 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:16:10.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:16:10.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:16:10.078 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:16:10.078 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 300 ceph tell osd.0 flush_pg_stats 2026-02-12T19:16:10.158 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-12T19:16:10.158 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-12T19:16:10.159 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: seqs=' 0-21474836485' 2026-02-12T19:16:10.159 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: for s in $seqs 2026-02-12T19:16:10.159 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: echo 0-21474836485 2026-02-12T19:16:10.159 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: cut -d - -f 1 2026-02-12T19:16:10.160 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: osd=0 2026-02-12T19:16:10.161 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-12T19:16:10.161 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 2 2026-02-12T19:16:10.162 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: seq=21474836485 2026-02-12T19:16:10.162 INFO:tasks.workunit.client.0.trial010.stdout:waiting osd.0 seq 21474836485 2026-02-12T19:16:10.163 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-12T19:16:10.163 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:16:10.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-12T19:16:10.382 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:16:11.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-12T19:16:11.384 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:16:11.602 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-12T19:16:11.602 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: sleep 1 2026-02-12T19:16:12.603 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-12T19:16:12.603 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-12T19:16:12.816 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: test 21474836486 -lt 21474836485 2026-02-12T19:16:12.817 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-12T19:16:12.817 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-12T19:16:12.817 INFO:tasks.workunit.client.0.trial010.stdout:#---------- 1.0 loop 0 2026-02-12T19:16:12.817 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-12T19:16:12.817 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-12T19:16:12.817 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-12T19:16:12.817 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-12T19:16:12.817 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-12T19:16:12.903 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state=active+clean 2026-02-12T19:16:12.903 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ active+clean == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-12T19:16:12.903 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: break 2026-02-12T19:16:12.903 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1719: wait_for_pg_clean: return 0 2026-02-12T19:16:12.903 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: get_last_scrub_stamp 1.0 2026-02-12T19:16:12.903 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-12T19:16:12.903 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-12T19:16:12.904 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-12T19:16:12.904 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-12T19:16:13.122 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1926: pg_scrub: local last_scrub=2026-02-12T19:16:05.661307+0000 2026-02-12T19:16:13.123 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1927: pg_scrub: ceph pg scrub 1.0 2026-02-12T19:16:13.322 INFO:tasks.workunit.client.0.trial010.stderr:instructing pg 1.0 on osd.0 to scrub 2026-02-12T19:16:13.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1928: pg_scrub: wait_for_scrub 1.0 2026-02-12T19:16:05.661307+0000 2026-02-12T19:16:13.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2063: wait_for_scrub: local pgid=1.0 2026-02-12T19:16:13.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2064: wait_for_scrub: local last_scrub=2026-02-12T19:16:05.661307+0000 2026-02-12T19:16:13.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2065: wait_for_scrub: local sname=last_scrub_stamp 2026-02-12T19:16:13.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i=0 )) 2026-02-12T19:16:13.332 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-12T19:16:13.332 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-12T19:16:13.333 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-12T19:16:13.333 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-12T19:16:13.333 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-12T19:16:13.333 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-12T19:16:13.547 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-12T19:16:05.661307+0000 '>' 2026-02-12T19:16:05.661307+0000 2026-02-12T19:16:13.547 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-12T19:16:14.548 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-12T19:16:14.548 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-12T19:16:14.548 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-12T19:16:14.548 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-12T19:16:14.548 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-12T19:16:14.549 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-12T19:16:14.549 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-12T19:16:14.762 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-12T19:16:05.661307+0000 '>' 2026-02-12T19:16:05.661307+0000 2026-02-12T19:16:14.762 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2071: wait_for_scrub: sleep 1 2026-02-12T19:16:15.763 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i++ )) 2026-02-12T19:16:15.763 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2067: wait_for_scrub: (( i < 300 )) 2026-02-12T19:16:15.764 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: get_last_scrub_stamp 1.0 last_scrub_stamp 2026-02-12T19:16:15.764 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1517: get_last_scrub_stamp: local pgid=1.0 2026-02-12T19:16:15.764 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1518: get_last_scrub_stamp: local sname=last_scrub_stamp 2026-02-12T19:16:15.764 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1519: get_last_scrub_stamp: ceph --format json pg dump pgs 2026-02-12T19:16:15.764 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1520: get_last_scrub_stamp: jq -r '.pg_stats | .[] | select(.pgid=="1.0") | .last_scrub_stamp' 2026-02-12T19:16:15.983 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2068: wait_for_scrub: test 2026-02-12T19:16:13.959979+0000 '>' 2026-02-12T19:16:05.661307+0000 2026-02-12T19:16:15.983 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2069: wait_for_scrub: return 0 2026-02-12T19:16:15.983 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1950: test_pg_scrub: kill_daemons td/ceph-helpers KILL osd 2026-02-12T19:16:15.984 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T19:16:15.984 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T19:16:15.984 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T19:16:15.984 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T19:16:15.984 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T19:16:16.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T19:16:16.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: WAIT_FOR_CLEAN_TIMEOUT=10 2026-02-12T19:16:16.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: TIMEOUT=2 2026-02-12T19:16:16.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1951: test_pg_scrub: pg_scrub 1.0 2026-02-12T19:16:16.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1923: pg_scrub: local pgid=1.0 2026-02-12T19:16:16.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1925: pg_scrub: wait_for_pg_clean 1.0 2026-02-12T19:16:16.091 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1702: wait_for_pg_clean: local pg_id=1.0 2026-02-12T19:16:16.091 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: get_timeout_delays 10 1 3 2026-02-12T19:16:16.092 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-12T19:16:16.092 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-12T19:16:16.092 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-12T19:16:16.092 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-12T19:16:16.092 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-12T19:16:16.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: delays=('1' '2' '3' '3' '1') 2026-02-12T19:16:16.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1703: wait_for_pg_clean: local -a delays 2026-02-12T19:16:16.127 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1704: wait_for_pg_clean: local -i loop=0 2026-02-12T19:16:16.128 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1706: wait_for_pg_clean: flush_pg_stats 2026-02-12T19:16:16.128 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=2 2026-02-12T19:16:16.128 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-12T19:16:16.351 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-12T19:16:16.351 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-12T19:16:16.351 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-12T19:16:16.352 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: timeout 2 ceph tell osd.0 flush_pg_stats 2026-02-12T19:16:18.357 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-12T19:16:18.357 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: true 2026-02-12T19:16:18.357 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-12T19:16:18.357 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: continue 2026-02-12T19:16:18.357 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-12T19:16:18.357 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 0' 2026-02-12T19:16:18.358 INFO:tasks.workunit.client.0.trial010.stdout:#---------- 1.0 loop 0 2026-02-12T19:16:18.358 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-12T19:16:18.358 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-12T19:16:18.358 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-12T19:16:18.358 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-12T19:16:18.358 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-12T22:12:42.262 DEBUG:teuthology.orchestra.run:got remote process result: 124 2026-02-12T22:12:42.270 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: pg_state= 2026-02-12T22:12:42.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1: is_pg_clean: teardown td/ceph-helpers 1 2026-02-12T22:12:42.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ceph-helpers 2026-02-12T22:12:42.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=1 2026-02-12T22:12:42.270 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ceph-helpers KILL 2026-02-12T22:12:42.270 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-12T22:12:42.271 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-12T22:12:42.271 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-12T22:12:42.271 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-12T22:12:42.271 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-12T22:12:42.365 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-12T22:12:42.366 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-12T22:12:42.366 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-12T22:12:42.367 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-12T22:12:42.368 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-12T22:12:42.368 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-12T22:12:42.369 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-12T22:12:42.370 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T22:12:42.370 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-12T22:12:42.371 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-12T22:12:42.371 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T22:12:42.372 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-12T22:12:42.374 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 1 = 1 ']' 2026-02-12T22:12:42.374 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:181: teardown: '[' -n '' ']' 2026-02-12T22:12:42.374 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:185: teardown: mkdir -p /home/ubuntu/cephtest/archive/log 2026-02-12T22:12:42.375 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:186: teardown: mv td/ceph-helpers/mgr.x.log td/ceph-helpers/mon.a.log td/ceph-helpers/osd.0.log /home/ubuntu/cephtest/archive/log 2026-02-12T22:12:42.377 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ceph-helpers 2026-02-12T22:12:42.381 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-12T22:12:42.381 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-12T22:12:42.381 INFO:tasks.workunit.client.0.trial010.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.52248 2026-02-12T22:12:42.381 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.52248 2026-02-12T22:12:42.383 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-12T22:12:42.383 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-12T22:12:42.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1575: is_pg_clean: [[ '' == \a\c\t\i\v\e\+\c\l\e\a\n* ]] 2026-02-12T22:12:42.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1711: wait_for_pg_clean: (( 0 >= 5 )) 2026-02-12T22:12:42.383 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1716: wait_for_pg_clean: sleep 1 2026-02-12T22:12:43.384 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1717: wait_for_pg_clean: loop+=1 2026-02-12T22:12:43.384 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1708: wait_for_pg_clean: true 2026-02-12T22:12:43.384 INFO:tasks.workunit.client.0.trial010.stdout:#---------- 1.0 loop 1 2026-02-12T22:12:43.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1709: wait_for_pg_clean: echo '#---------- 1.0 loop 1' 2026-02-12T22:12:43.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1710: wait_for_pg_clean: is_pg_clean 1.0 2026-02-12T22:12:43.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1572: is_pg_clean: local pgid=1.0 2026-02-12T22:12:43.385 INFO:tasks.workunit.client.0.trial010.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1573: is_pg_clean: local pg_state 2026-02-12T22:12:43.385 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: ceph pg 1.0 query 2026-02-12T22:12:43.385 INFO:tasks.workunit.client.0.trial010.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1574: is_pg_clean: jq -r '.state ' 2026-02-12T22:13:42.270 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2026-02-12T22:14:42.270 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: 2026-02-12T22:14:42.271 INFO:tasks.workunit:Stopping ['misc'] on client.0... 2026-02-12T22:14:42.271 DEBUG:teuthology.orchestra.run.trial010:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-12T22:14:42.599 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks/workunit.py", line 144, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks/workunit.py", line 292, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_afefaa40503a4bf024e96f97dd7f852b75f00ad6/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/remote.py", line 575, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test misc/test-ceph-helpers.sh) on trial010 with status 124: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh' 2026-02-12T22:14:42.604 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-12T22:14:42.619 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-12T22:14:42.619 DEBUG:teuthology.orchestra.run.trial010:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-12T22:14:42.725 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-12T22:14:42.736 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-12T22:14:42.736 DEBUG:teuthology.orchestra.run.trial010:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T22:14:42.785 INFO:teuthology.orchestra.run.trial010.stdout: remote refid st t when poll reach delay offset jitter 2026-02-12T22:14:42.785 INFO:teuthology.orchestra.run.trial010.stdout:============================================================================== 2026-02-12T22:14:42.785 INFO:teuthology.orchestra.run.trial010.stdout:+time.cloudflare 10.138.8.13 3 u 139 256 377 3.056 +0.862 0.192 2026-02-12T22:14:42.785 INFO:teuthology.orchestra.run.trial010.stdout:*73.185.182.209 204.117.185.216 2 u 135 256 377 41.968 -0.720 1.178 2026-02-12T22:14:42.785 INFO:teuthology.orchestra.run.trial010.stdout:-sushi.ruselabs. 2.148.140.233 2 u 247 256 377 66.989 +3.829 1.703 2026-02-12T22:14:42.785 INFO:teuthology.orchestra.run.trial010.stdout:+y.ns.gin.ntt.ne 129.250.35.222 2 u 188 256 377 3.167 +1.171 2.667 2026-02-12T22:14:42.785 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-12T22:14:42.797 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-12T22:14:42.797 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-12T22:14:42.811 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-12T22:14:42.827 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-12T22:14:42.843 INFO:teuthology.task.internal:Duration was 11462.919074 seconds 2026-02-12T22:14:42.843 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-12T22:14:42.857 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-12T22:14:42.858 DEBUG:teuthology.orchestra.run.trial010:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-12T22:14:42.883 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-12T22:14:42.883 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial010.front.sepia.ceph.com 2026-02-12T22:14:42.883 DEBUG:teuthology.orchestra.run.trial010:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-12T22:14:42.932 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-12T22:14:42.933 DEBUG:teuthology.orchestra.run.trial010:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T22:14:43.110 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-12T22:14:43.110 DEBUG:teuthology.orchestra.run.trial010:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-12T22:14:43.119 INFO:teuthology.orchestra.run.trial010.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T22:14:43.120 INFO:teuthology.orchestra.run.trial010.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T22:14:43.120 INFO:teuthology.orchestra.run.trial010.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-12T22:14:43.120 INFO:teuthology.orchestra.run.trial010.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T22:14:43.120 INFO:teuthology.orchestra.run.trial010.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-12T22:14:43.160 INFO:teuthology.orchestra.run.trial010.stderr: 82.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-12T22:14:43.162 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-12T22:14:43.177 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-12T22:14:43.178 DEBUG:teuthology.orchestra.run.trial010:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-12T22:14:43.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-12T22:14:43.238 DEBUG:teuthology.orchestra.run.trial010:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-12T22:14:43.266 INFO:teuthology.orchestra.run.trial010.stdout:kernel.core_pattern = core 2026-02-12T22:14:43.276 DEBUG:teuthology.orchestra.run.trial010:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-12T22:14:43.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T22:14:43.321 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-12T22:14:43.337 INFO:teuthology.task.internal:Transferring archived files... 2026-02-12T22:14:43.340 DEBUG:teuthology.misc:Transferring archived files from trial010:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-12_15:38:14-rados-wip-yuri4-testing-2026-02-11-2126-distro-default-trial/46611/remote/trial010 2026-02-12T22:14:43.341 DEBUG:teuthology.orchestra.run.trial010:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-12T22:14:56.922 INFO:teuthology.task.internal:Removing archive directory... 2026-02-12T22:14:56.922 DEBUG:teuthology.orchestra.run.trial010:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-12T22:14:57.019 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-12T22:14:57.030 INFO:teuthology.task.internal:Not uploading archives. 2026-02-12T22:14:57.030 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-12T22:14:57.044 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-12T22:14:57.045 DEBUG:teuthology.orchestra.run.trial010:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-12T22:14:57.049 INFO:teuthology.orchestra.run.trial010.stdout: 1180220 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 12 22:14 /home/ubuntu/cephtest 2026-02-12T22:14:57.049 INFO:teuthology.orchestra.run.trial010.stdout: 1190960 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 12 19:05 /home/ubuntu/cephtest/mnt.0 2026-02-12T22:14:57.049 INFO:teuthology.orchestra.run.trial010.stdout: 1190961 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 12 19:12 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-12T22:14:57.049 INFO:teuthology.orchestra.run.trial010.stdout: 1206853 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 12 19:12 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-12T22:14:57.049 INFO:teuthology.orchestra.run.trial010.stdout: 1206854 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Feb 12 22:12 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td 2026-02-12T22:14:57.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T22:14:57.050 INFO:teuthology.orchestra.run.trial010.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-12T22:14:57.050 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_8bec0da71becad44414c54979f64c9ef0e7099c6/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial010 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-12T22:14:57.051 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-12T22:14:57.063 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-12T22:14:57.077 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial010 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-12T22:14:57.081 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/misc} duration: 11462.919074058533 failure_reason: 'Command failed (workunit test misc/test-ceph-helpers.sh) on trial010 with status 124: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afefaa40503a4bf024e96f97dd7f852b75f00ad6 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/misc/test-ceph-helpers.sh''' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-12T22:14:57.081 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T22:14:57.188 INFO:teuthology.run:FAIL